This art icle was downloaded by: [ North Dakota State University]On: 20 March 2012, At : 09: 29Publisher: Rout ledgeI nform a Ltd Registered in England and Wales Registered Num ber: 1072954 Registeredoffice: Mort im er House, 37-41 Mort im er St reet , London W1T 3JH, UK

Research in Science & TechnologicalEducationPublicat ion details, including inst ruct ions for authors andsubscript ion informat ion:ht tp:/ / www.tandfonline.com/ loi/ crst20

A cross-cultural validation of theTechnology-Rich Outcomes-FocusedLearning Environment Inventory(TROFLEI) in Turkey and the USAAnita G. Welch a , Mustafa Cakir b , Claudet te M. Peterson a &

Chris M. Ray a

a School of Educat ion, North Dakota State University, Fargo, NorthDakota, USAb Atatürk Faculty of Educat ion, Marmara University, Marmara,Turkey

Available online: 19 Mar 2012

To cite this article: Anita G. Welch, Mustafa Cakir, Claudet te M. Peterson & Chris M. Ray (2012): Across-cultural validat ion of the Technology-Rich Outcomes-Focused Learning Environment Inventory(TROFLEI) in Turkey and the USA, Research in Science & Technological Educat ion, 30:1, 49-63

To link to this article: ht tp:/ / dx.doi.org/ 10.1080/ 02635143.2012.659179

PLEASE SCROLL DOWN FOR ARTI CLE

Full term s and condit ions of use: ht tp: / / www.tandfonline.com / page/ term s-and-condit ions

This art icle m ay be used for research, teaching, and private study purposes. Anysubstant ial or system at ic reproduct ion, redist r ibut ion, reselling, loan, sub- licensing,system at ic supply, or dist r ibut ion in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or im plied or m ake any representat ionthat the contents will be com plete or accurate or up to date. The accuracy of anyinst ruct ions, form ulae, and drug doses should be independent ly verified with pr im arysources. The publisher shall not be liable for any loss, act ions, claim s, proceedings,

dem and, or costs or dam ages whatsoever or howsoever caused arising direct ly orindirect ly in connect ion with or ar ising out of the use of this m aterial.

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

A cross-cultural validation of the Technology-Rich Outcomes-

Focused Learning Environment Inventory (TROFLEI) in Turkey

and the USA

Anita G. Welcha*, Mustafa Cakirb, Claudette M. Petersona and Chris M. Raya

aSchool of Education, North Dakota State University, Fargo, North Dakota, USA; bAtatürkFaculty of Education, Marmara University, Marmara, Turkey

Background: Studies exploring the relationship between students’ achievementand the quality of the classroom learning environments have shown that there isa strong relationship between these two concepts. Learning environment instru-ments are constantly being revised and updated, including for use in differentcultures, which requires continued validation efforts.Purpose: The purpose of this study was to establish cross-cultural reliability andvalidity of the Technology-Rich Outcomes-Focused Learning EnvironmentInventory (TROFLEI) in both Turkey and the USA.Sample: Approximately 980 students attending grades 9–12 in Turkey and 130students attending grades 9–12 in the USA participated in the study.Design and method: Scale reliability analyses and confirmatory factor analysis(CFA) were performed separately for Turkish and US participants for bothactual and preferred responses to each scale to confirm the structure of theTROFLEI across these two distinct samples.Results: Cronbach’s alpha reliability coefficients, ranging from α = 0.820 to0.931 for Turkish participants and from α = 0.778 to 0.939 for US participants,indicated that all scales have satisfactory internal consistency for both samples.Confirmatory factor analyses resulted in evidence of adequate model fit acrossboth samples for both actual and preferred responses, with the root mean squareerror of approximation ranging from 0.052 to 0.057 and the comparative fitindex ranging from 0.920 to 0.982.Conclusions: This study provides initial evidence that the TROFLEI is valid foruse in both the Turkish and US high-school populations (grades 9–12). How-ever, the psychometric properties should be examined further with different pop-ulations, such as middle-school students (grades 6–8).

Keywords: TROFLEI; cross-cultural validation; classroom environment; tech-nology-rich classroom; confirmatory factor analysis

Introduction

Studies assessing the relationship between students’ achievement and the quality ofthe classroom learning environment (Goh 2002; Khine 2002) have shown that thereis a strong relationship between these two concepts (Fraser 2007). When discussingthe importance of researching classroom learning environments, Fraser stated thatthe reason is simple: ‘because students spend approximately 20,000 hours inclassrooms by the time that they graduate from university’ (Fraser 2001, 1). From a

*Corresponding author. Email: anita.welch@ndsu.edu

Research in Science & Technological Education

Vol. 30, No. 1, April 2012, 49–63

ISSN 0263-5143 print/ISSN 1470-1138 online� 2012 Taylor & Francishttp://dx.doi.org/10.1080/02635143.2012.659179http://www.tandfonline.com

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

psychological view, students learn better when they perceive the classroom environ-ment more positively (Dorman, Aldridge, and Fraser 2006; Dorman 2009). Accord-ing to Fraser (2002), researchers around the world pay great attention to studentachievement but little attention to the actual environment of the classrooms.

There is an urgent need for instruments that assess and evaluate learning envi-ronments in Turkey. Den Brok et al. (2010) reported that learning environmentresearch in Turkey should focus on developing instruments for measuring students’and teachers’ perceptions of learning environments and use such knowledge inorder to improve the instructional practices. Instruction and learning can beimproved with a systematic review and evaluation of learning environments (Hof-stein, Nahum, and Shore 2001). Using instruments directly in different countries isalmost impossible because of the constraints on spoken language, cultural andsocial differences. In this respect, MacLeod and Fraser (2010) indicated that transla-tions of validated learning environment questionnaires have provided valuable toolsfor researchers in many countries.

Learning environment instruments are constantly being revised and updated.One such current effort was to develop and validate the Technology-Rich Out-comes-Focused Learning Environment Inventory (TROFLEI) (Aldridge, Dorman,and Fraser 2004) based upon an expansion of an existing instrument. The purposeof this study was to investigate and establish the cross-cultural reliability and valid-ity of the TROFLEI for use with high-school students in both Turkey and the USA.

Theoretical framework

Research on learning environments

Goh (2002) identified student cohesiveness, self-esteem and confidence, motivationand a sense of belonging as psychosocial dimensions that influence the learningenvironment. Moreover, Heartel, Walberg, and Heartel (1981) indicated that satis-faction, goal direction, difficulty, competitiveness and friction are social–psychologi-cal dimensions that also influence the learning environment. Thus, the promotion ofthe positive effect of these psychosocial dimensions depends on well-organizedclassrooms where supportive relationships with teachers and classmates are formedand where there is an emphasis on participation (Moos 1991). Creating a classroomenvironment where students learn meaningfully depends on psychologically appro-priate and supportive classrooms. There is no doubt that assessment of the class-room environment provides us a clue to how classrooms should be organized.

Assessment of learning environments

Researchers have developed many instruments to assess perceptions of learningenvironments over the past 40 years. In this process, the attention on the study ofclassroom learning environments has grown increasingly. According to Dorman(2003), the use of valid instruments is essential to the conduct of meaningfulresearch; these instruments have been validated in numerous studies by a number ofresearchers.

As already mentioned, using these instruments directly in different countries isalmost impossible not only because of the constraints on spoken language but alsobecause of national boundaries such as cultural and social differences. In thisrespect, MacLeod and Fraser (2010) indicated that translations of validated learning

50 A.G. Welch et al.

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

environment questionnaires have provided valuable tools for researchers in manyparts of the world. Cross-national studies in education provide a wide perspectiveon generating new insights, for two main reasons (Fraser 1991, 2002; Fraser,Aldridge, and Adolphe 2010). First, there is usually more variation in variables ofinterest such as teaching methods and student attitudes in a sample drawn frommultiple countries than from a one-country sample. Second, the comparison of edu-cational practices, beliefs and attitudes of two countries provides deep insights intotheir relative positions with regard to education. Thus, the need for translations ofpopular questionnaires from the original language to other languages has expandedrecently (MacLeod and Fraser 2010).

One of the most functional learning environment questionnaires is the Technol-ogy-Rich Outcomes-Focused Learning Environment Inventory (TROFLEI), devel-oped by Aldridge and Fraser (2003). TROFLEI is a 10-scale instrument that‘investigates how information and communication technologies can be used effec-tively to maximize educational outcomes for individual students’ (Clayton 2007,40). This instrument is an expansion of Fraser, Fisher, and McRobbie’s (1996)What Is Happening In This Class (WIHIC) instrument, which served as a measureof classroom climate. After the analysis of data from an Australian sample of 1081students in 50 classes conducted by Aldridge and Fraser (2000), the final form ofthe WIHIC was generated with seven eight-item scales (Fraser 2002). The TROF-LEI includes seven scales of WIHIC (student cohesiveness, teacher support,involvement, investigation, task orientation, cooperation and equity) and three newscales (differentiation, computer usage and young adult ethos) that include a focuson technology and outcomes of secondary school classrooms (Aldridge and Fraser2003; Aldridge, Dorman, and Fraser 2004).

Besides the USA, Canada, Australia and Britain, the WIHIC is frequently usedin Asia. The instrument has been used successfully in Singapore (Khoo and Fraser1998; Chionh and Fraser 2009), Taiwan (Aldridge and Fraser 2000; Aldridge,Fraser, and Huang 1999), Brunei (Khine and Fisher 2001; Riah and Fraser 1998),Indonesia (Margianti, Fraser, and Aldridge 2001; Soerjaningsih, Fraser, and Ald-ridge 2001), Korea (Kim, Fisher, and Fraser 2000), Turkey (Telli, Çakıroğlu, andBrok 2006) and Dubai (Macleod and Fraser 2010). The WIHIC has been translatedinto seven languages, namely, Chinese, Mandarin, Indonesian, Korean, Spanish,Turkish and Arabic.

Few studies about learning environments have been conducted in Turkey. Oneof these studies investigates the associations between students’ perceptions of theirlearning environments and their attitudes towards biology (Telli, Çakıroğlu andBrok 2006). The researchers used WIHIC and another attitude questionnaire, theTest of Science Related Attitudes (TOSRA), to gather data from 1983 students in57 classrooms in two major Turkish cities. Telli and her colleagues used samemethodology for translation and back-translation as in other examples. Scale reli-ability ranged from 0.75 to 0.88 for different WIHIC scales. The findings of thisresearch indicated that students’ perceptions about the learning environment weresignificantly correlated with their attitudes towards biology.

Aldridge and Fraser (2003) indicated that there has been a lack of informationabout the implementation and educational benefits of technology-rich school learn-ing environments. Because of this, they generated TROFLEI, which can be used toassess the impact of the integration of information and communication technology(ICT) on the learning environment. The questionnaire has two versions; an actual

Research in Science & Technological Education 51

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

version, which records what students perceive as happening in the classroom, and apreferred version, which records what students would prefer to occur in their class.They have also developed a second instrument for students’ attitudes through astudy with 1035 students in 80 classes. The purposes of the research were: (1) tovalidate a widely applicable questionnaire for monitoring outcomes-focused andICT-rich classroom learning environments and student attitudes; (2) to investigatewhether outcomes-focused and ICT-rich learning environments promote studentachievement, attitudes towards the subject, attitudes to use of ICT and academicefficacy; and (3) to evaluate the success of educational programs at an innovativenew school in terms of changes in learning environment and student attitudes. Theresearchers performed factor analysis and item analysis for the purpose of validationof the questionnaire. Originally the TROFLEI contained 10 eight-item scales. Afterthe validation process, a refined version of the questionnaire consisted 77 items inthe same 10 scales. The items were measured with five-point scales ranging fromalmost never to almost always.

Aldridge and Fraser (2003) suggested that this study could have significantimplications for educational systems concerning how ICT can be used effectivelyto maximize educational outcomes for individual students. Aldridge, Dorman, andFraser (2004) also studied the development and validation of TROFLEI, employ-ing multitrait–multimethod modeling (MTMM) within a confirmatory factor analy-sis (CFA) framework. There are numerous MTMM models suggested in theliterature, such as general confirmatory factor analysis, Campbell–Fiske, correlateduniqueness confirmatory factor analysis and the composite direct product (Byrneand Goffin 1993). The authors have chosen general CFA because of this model’sprevalence in measuring psychological constructs. Additionally, Byrne and Goffin(1993) suggested that MTMM has provided an essential assessment tool for con-struct validation.

Aldridge and colleagues (2004) administered TROFLEI to 772 students from 79classes in Western Australia and 477 students from 48 classes in Tasmania. Eachstudent was asked to respond to the actual and preferred forms of the questionnaire.The researchers computed Cronbach’s alpha coefficient for internal consistency reli-ability, mean correlation between scales for discriminant validity, and mean andstandard deviation scores for descriptive results. Next, separate exploratory factoranalyses (EFA) on the actual and preferred forms of the TROFLEI were conductedand, finally, MTMM analyses within a CFA framework were used to explore theconstruct validity.

Dorman (2009) used TROFLEI to investigate associations between classroomenvironment and student outcomes (academic efficacy, attitude to subject and atti-tude to computer usage) in Australian secondary schools. He undertook this studyto validate TROFLEI and also to validate attitude scales for academic efficacy, atti-tude to subject and attitude to computer usage. Dorman administered the TROFLEIand the three additional scales as indicated earlier to 4407 students in 286 classes.The results from this study indicated that students’ perceptions of the classroomenvironment were significantly related to their affective outcomes.

Overview and goals of this study

The goal of the study reported here was to investigate the cross-cultural validity ofthe TROFLEI, as designed by Dorman and Fraser (2009), for both American and

52 A.G. Welch et al.

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

Turkish high-school students. This will allow the comparison of classroom environ-ments across these two cultures with classrooms in other cultures where the instru-ment has been validated previously.

Method

Translation procedure

To accomplish the study’s purpose, the TROFLEI first had to be translated intoTurkish according to established guidelines. The translation process was a combina-tion of the three translation protocols. Brislin (1970) noted that it is acceptable tocombine more than one translation technique within the same research project. Hemade this suggestion at a time when the tendency was not to use any particular pro-tocol. He reported a review of 80 articles published in the same year (1970), indi-cating that more than 75% of the articles gave little or no information that wouldallow a reader to rule out problems with translation. Some of the remaining articlesjust mentioned that a bilingual friend had helped the researcher to translate aninstrument from one language to another.

In the first step of this multistep process, the Turkish researcher and two col-leagues independently translated the instrument from the source language of Englishto the target language of Turkish. The researcher compared all three translations inthe target language and prepared a draft of the instrument in Turkish. He then sentthe original TROFLEI instrument, which was in English, along with the Turkishdraft to four other bilingual colleagues and asked whether they agreed with the trans-lation. They indicated for each item whether they agreed with the translation; if theydid not, they proposed an alternative in Turkish. All of these individuals have PhDsin Education from American or British universities and they work at colleges of edu-cation in different universities in Turkey. Finally, the Turkish researcher and anothercolleague assessed and discussed the responses and prepared the final version. Thisprocess was similar to the translation committee protocol, in which bilingual individ-uals work independently to translate the original text into the target language andthen collaborate to reach a consensus on a final translation (Brislin 1986).

The Turkish version of the instrument was strengthened by carrying out back-translation as recommended by Brislin (1976). In order to complete the back-trans-lation process, different people, independent of the project, who speak both Englishand Turkish were asked to translate the target Turkish version back into the sourcelanguage of English without having the original English version to influence theirtranslation. These translators were not involved with the initial source to targettranslation.

In the end, three individuals who were in three different regions of the USAworking independently were asked to translate the Turkish version of the instrumentback into English. Although they were not teachers of children in the target popula-tion grade, they were asked to keep in mind the ages of the students who would beparticipating in order to ensure that the level of complexity of language was suitablefor that age group. As each of the individuals returned their translation, the Ameri-can researcher combined them all into a single table. A portion of this table isshown in Figure 1. This allowed for easy comparison of the various versions. Thiscomparison was performed by a monolingual English-speaking researcher, becauseit can be problematic for the final comparison to be made by bilingual individuals(Sireci and Berberoġlu 2000).

Research in Science & Technological Education 53

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

There were no major discrepancies between the different versions, or betweenthe original TROFLEI instrument in English and the version that resulted from theback-translation. However, there was a distinct different between the ‘Thank you’message from the original instrument and the back-translated version as shown inFigure 2. The source version simply says ‘Thank you for your assistance in com-pleting this questionnaire’, while the translation references truth, sincerity and hon-esty. Given that this statement was presented upon conclusion of data collection,however, the difference was deemed to be inconsequential.

The resulting items for both the Turkish translation and the English version ofthe TROFLEI are shown in Appendix 1.

Sample

In Turkey, the researcher contacted the local school district authority in Istanbul andsurrounding areas and received permission to conduct the study. Table 1 shows thenumber of students in each grade in the Turkish sample.

In the USA, the researchers posted a request on the National Association forResearch in Science Teaching (NARST) listserv and a listserv managed by theAmerican Educational Research Association (AERA) looking for science teacherswith high-school students (grades 9–12) in science classes who would be willing tohave their students complete the TROFLEI instrument. Teachers responded frommany states, providing a representative sample of students from large metropolitanareas to small rural locations. Table 2 shows the number of students in each gradein the US sample.

In both countries, the TROFLEI was administered to the students electronicallyusing an online survey tool.

Data analysis

To investigate the cross-cultural validity of Dorman and Fraser’s (2009) TROFLEIwhen used with American and Turkish high-school students, scale reliability analy-ses and a CFA were performed to confirm the structure of the TROFLEI.

Figure 2. Sample of last entry in the translation comparison table.

Figure 1. Sample of table used to compare different versions of back-translation of theTROFLEI instrument.

54 A.G. Welch et al.

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

Results

Scale reliability analysis

Cronbach’s alpha (α) reliability coefficients were calculated for each scale. Theresults indicated that all scales have satisfactory internal consistency. The indicesfor the Turkish actual and preferred scales ranged from 0.820 to 0.920 and 0.842 to0.931, respectively, and are similar to those reported in previous research (e.g. Dor-man and Fraser 2009; Dorman, Aldridge, and Fraser 2006). Indices for the USactual and preferred scales ranged from 0.778 to 0.919 and 0.811 to 0.939, respec-tively. The differentiation scale in the US actual instrument was slightly lower thanthe 0.82 reported by Dorman and Fraser (2009), but is still within acceptable limits.Table 3 shows the reliability statistics for each version and each scale.

Confirmatory factor analysis

A maximum-likelihood CFA was performed using LISREL 8.8 (Jöreskog and Sör-bom 2006) on the data from both Turkey and the USA to identify the structure ofthe 80-item TROFLEI. Separate analyses were conducted on the actual and pre-ferred data for each sample, using the 10 first-order latent variables of the TROF-LEI and the 80 observed variables. Numerous fit indices were consideredsimultaneously to evaluate model fit, including the observed chi-square values, thecomparative fit index (CFI) and the root mean square error of approximation(RMSEA). Results are shown in Table 4.

For this study, a combination of χ2/df ratio less than 3, CFI greater than 0.90

and RMSEA less than 0.06 was considered to be ideal (Hu and Bentler 1999).While it is possible for some indices to indicate fit while others will indicate aslight lack of fit owing to the differing mathematical foundations for each index,the majority of the indices were expected to meet the established criteria while theremaining indices were expected to be close to these criteria (Marsh, Hau, and Wen2004). As a result, the results generally indicate a good fit to the data and provideinitial evidence to support the 10-scale structure of the TROFLEI in both countries.

Table 2. Description of the US sample.

n

Gender Grade 9 Grade 10 Grade 11 Grade 12

Male 12 18 10 11Female 22 29 15 14Total 34 47 25 25

Table 1. Description of the Turkish sample.

n

Gender Grade 9 Grade 10 Grade 11 Grade 12

Male 99 163 92 92Female 91 189 125 134Total 190 352 217 226

Research in Science & Technological Education 55

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

While the CFI and RMSEA indices for the US sample are not as high, which islikely to be due to the smaller sample size, the WIHIC upon which the TROFLEIis based has been previously validated for US participants.

Discussion

The goal of this study was to investigate the cross-cultural validity of the TROFLEI(Dorman and Fraser 2009) when used with American and Turkish high-school

Table 3. Internal consistency reliability and scale statistics.

Turkish actual Turkish preferred

Scale α Mean Variance α Mean Variance

SC 0.820 0.926 0.097 0.849 4.492 0.032TS 0.920 3.445 0.099 0.931 4.435 0.014IN 0.869 3.637 0.024 0.886 4.184 0.022TO 0.866 3.705 0.020 0.910 4.282 0.018IV 0.863 4.038 0.057 0.919 4.585 0.005CO 0.892 3.438 0.076 0.910 4.068 0.044EQ 0.917 4.124 0.026 0.928 4.556 0.006DI 0.810 3.240 0.182 0.842 3.875 0.167CU 0.844 3.169 0.526 0.883 3.896 0.157YA 0.886 4.024 0.051 0.902 4.482 0.008

US actual US preferred

Scale α Mean Variance α Mean Variance

SC 0.815 3.976 0.155 0.851 4.221 0.112TS 0.919 3.420 0.110 0.898 3.925 0.101IN 0.864 3.285 0.068 0.895 3.576 0.023TO 0.906 3.156 0.032 0.939 3.448 0.015IV 0.821 4.259 0.018 0.811 4.580 0.005CO 0.873 3.907 0.062 0.884 4.169 0.026EQ 0.900 4.150 0.022 0.887 4.416 0.007DI 0.778 2.910 0.488 0.808 3.226 0.477CU 0.863 2.580 0.403 0.900 3.088 0.201YA 0.906 4.216 0.032 0.888 4.421 0.015

Note: SC: Student Cohesiveness; TS: Teacher Support; IN: Involvement; TO: Task Orientation; IV:Investigation; CO: Cooperation; EQ: Equity; DI: Differentiation; CU: Computer Usage; YA: YoungAdult Ethos.

Table 4. Goodness-of-fit statistics.

Model df χ2

90% CI forRMSEA RMSEA

p-ValueRMSEA CFI

Turkish actual 3035.000 9103.287 0.0511; 0.0532 0.052 0.000 0.971Turkishpreferred

3035.000 9788.694 0.0527; 0.0548 0.054 0.000 0.982

US actual 3035.000 5066.054 0.0513; 0.0592 0.055 0.000 0.920US preferred 3035.000 5285.840 0.0535; 0.0612 0.057 0.000 0.929

Note: df: degrees of freedom; CI: confidence interval; RMSEA: root mean square error of approxima-tion; CFI: comparative fit index.

56 A.G. Welch et al.

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

students to extend the number of cultures for which the instrument is appropriate.The results of this study suggest that the TROFLEI can be used with both Turkishand US high-school students. With the Turkish sample, the TROFLEI showed goodreliability on both the actual (0.810 6 α 6 0.920) and preferred scales (0.892 6 α

6 0.931). The US sample also showed good reliability on both the actual (0.778 6α 6 0.919) and preferred scales (0.811 6 α 6 0.939). The CFA supported theunderlying theoretical model of the TROFLEI.

Although the results of this study provide a better understanding of the psycho-metric quality of the TROFLEI across Turkish and US populations, it is also impor-tant to highlight some limitations of the study. First, the study sample wascomposed of students in grades 9–12. Consequently, the sample is restricted basedon age and education level, which limits the generalizability of the results to a lar-ger population. Second, the majority of individuals were Turkish and these findingsmay not hold for other nationalities.

In summary, the results show that the TROFLEI is valid for use in both theTurkish and US high-school populations (grades 9–12). However, the psychometricproperties should be examined further in different populations, such as middle-school students (grades 6–8). Researchers need to be cognizant of the psychometricproperties of the instruments used and their cross-cultural equivalence. The increas-ing accessibility of cross-national data sets, such as the Trends in InternationalMathematics and Science Study (TIMSS), the Program for International Assessmentof Students’ Achievements (PISA) and the Progress in International Reading Liter-acy Study (PIRLS), allows researchers and practitioners to engage in cross-culturalanalysis more than ever before. The value of cross-cultural validation to allowresearchers and practitioners to compare classrooms across cultures is growing inimportance as a result of globalization as well as the increased use of global mea-sures of performance. Therefore, there is an increasing need for the evaluation ofinstruments such as the TROFLEI, to ensure that they measure the same constructsand yield comparable scores across the cultures of interest.

Acknowledgements

The authors would like to thank Dr Serhat İrez, Dr Hakan Akçay and Özgur KıvılcanDoğan from Marmara University, Dr Emine Ferda Bedel from Çanakkale Onsekiz MartUniversity, Dr Doğan Çömez from North Dakota State University, Dr Alper Kesten fromOndokuz Mayıs University, Dr Bülent Tarman from Selcuk University, Dr Yakut Gazi fromTexas A&M University and Dr Funda Şahin from the University of Tennessee for theirassistance with the translation of the TROFLEI. This research (in part) was supported byNational Science Foundation grant no. HRD-0811239 to the NDSU Advance FORWARDprogram.

References

Aldridge, J.M., J.P. Dorman, and B.J. Fraser. 2004. Use of multitrait-multimethod modelingto validate actual and preferred forms of the Technology-Rich Outcomes-Focused Learn-ing Environment Inventory (TROFLEI). Australian Journal of Educational & Develop-mental Psychology 4: 110–25.

Aldridge, J.M., and B.J. Fraser. 2000. A cross-cultural study of classroom learning environ-ments in Australia and Taiwan. Learning Environments Research 3: 101–34.

Aldridge, J.M., and B.J. Fraser. 2003. Effectiveness of a technology-rich and outcomes-focused learning environment. In Technology-rich learning environments: A future per-spective, ed. M.S. Khine and D. Fisher, 41–69. Singapore: World Scientific.

Research in Science & Technological Education 57

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

Aldridge, J.M., B.J. Fraser, and T. Huang. 1999. Investigating classroom environments inTaiwan and Australia with multiple research methods. Journal of Educational Research93: 48–57.

Brislin, R.W. 1970. Back-translation for cross-cultural research. Journal of Cross-CulturalResearch 1: 185–216.

Brislin, R.W. 1976. Comparative research methodology: Cross-cultural studies. InternationalJournal of Psychology 11: 215–29.

Brislin, R.W. 1986. The wording and translation of research instruments. In Field methodsin cross-cultural research, ed. W.J. Lonner and J.W. Berry, 137–64. Beverly Hills, CA:Sage.

den Brok, P., S. Telli, J. Çakıroğlu, T. Taconis, and C. Tekkaya. 2010. Learning environmentprofiles of Turkish secondary biology classrooms. Learning Environments Research 13:187–204.

Byrne, B., and R. Goffin. 1993. Modeling multitrait–multimethod data from additive andmultiplicative covariance structures: An audit of construct validity concordance. Multi-variate Behavioral Research 28: 67–96.

Chionh, Y.H., and B.J. Fraser. 2009. Classroom environment, achievement, attitudes andself-esteem in geography and mathematics in Singapore. International Research in Geo-graphical and Environmental Education 18: 29–44.

Clayton, J. 2007. Development and validation of an instrument for assessing online learningenvironments in tertiary education: The Online Learning Environment Survey (OLLES).Unpublished doctoral thesis, Curtin University of Technology, Perth.

Dorman, J.P. 2003. Cross-national validation of the What Is Happening In this Class?(WIHIC) questionnaire using confirmatory factor analysis. Learning EnvironmentResearch 6: 231–45.

Dorman, J.P. 2009. The effect of clustering on statistical tests conducted with classroomenvironment data. Australian Journal of Educational & Developmental Psychology 9:18–31.

Dorman, J.P., J.M. Aldridge, and B.J. Fraser. 2006. Using students’ assessment of classroomenvironment to develop a typology of secondary school classrooms. International Educa-tion Journal 7: 906–15.

Dorman, J.P., and B.J. Fraser. 2009. Psychosocial environment and affective outcomes intechnology-rich classrooms: testing a causal model. Social Psychology of Education: AnInternational Journal 12: 77–99.

Fraser, B.J. 1991. Two decades of classroom environment research. In Educational environ-ments: Evaluation antecedents and consequences, ed. B.J. Fraser and H.J. Walberg, 3–27. Oxford: Pergamon Press.

Fraser, B. 2001. Twenty thousand hours: Editor’s introduction. Learning EnvironmentsResearch: An International Journal 4: 1–5.

Fraser, B. J. 2002. Learning environments research: Yesterday today and tomorrow. In Stud-ies in educational learning environments, ed. S.C. Goh and J.B. Kahle, 49–72. Singa-pore: World Scientific Publishing.

Fraser, B.J. 2007. Classroom environments. In Handbook of research on science education, ed.S.K. Abell and N.G. Lederman, 102–24. Mahwah, NJ: Lawrence Erlbaum Associates.

Fraser, B.J., J.M. Aldridge, and F.S.G. Adolphe. 2010. A cross-national study of secondaryscience classroom environments in Australia and Indonesia. Research in Science Educa-tion 40: 551–71.

Fraser, B.J., D.L. Fisher, and C.J. McRobbie. 1996. Development, validation and use of per-sonal and class forms of a new classroom environment instrument. Paper presented atthe annual meeting of the American Educational Research Association, New York.

Goh, S.C. 2002. Studies on learning environments in Singapore classrooms. In Studies ineducational learning environments: An international perspective, ed. S.C. Goh and M.S.Khine, 1–26. Singapore: World Scientific.

Haertel, G.D., H.J. Walberg, and E.H. Haertel. 1981. Socio-psychological environments andlearning: A quantitative synthesis. British Educational Research Journal 7: 27–36.

Hofstein, A., T. Nahum, and R. Shore. 2001. Assessment of the learning environment ofinquiry-type laboratories in high school chemistry. Learning Environments Research 4:193–207.

58 A.G. Welch et al.

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

Hu, L., and R.M. Bentler. 1999. Cutoff criteria for fit indexes in covariance structure analy-sis: Conventional criteria versus new alternatives. Structural Equation Modeling 6, no. 1:1–55.

Jöreskog, K.G., and D. Sörbom. 2006. LISREL 8.8 for Windows [computer software]. Lin-colnwood, IL: Scientific Software International.

Khine, M.S. 2002. Study of learning environment for improving science education in Brunei.In Studies in educational learning environments, ed. S.C. Goh and J.B. Kahle, 49–72.Singapore: World Scientific.

Khine, M.S., and D.L. Fisher. 2001. Classroom environment and teachers’ cultural back-ground in secondary science classes in an Asian context. Paper presented at the annualconference of the Australian Association for Research in Education, Fremantle, WesternAustralia.

Khoo, H.S., and B.J. Fraser. 1998. Using classroom environment dimensions in the evalua-tion of adult computer courses. Paper presented at the annual meeting of the AmericanEducational Research Association, San Diego, CA.

Kim, H-B., D.L. Fisher, and B.J. Fraser. 2000. Classroom environment and teacher interper-sonal behaviour in secondary science Classes in Korea. Evaluation & Research in Edu-cation 14: 3–22.

MacLeod, C., and B.J. Fraser. 2010. Development, validation, and application of a modifiedArabic translation of the What Is Happening In this Class? (WIHIC) questionnaireLearning Environment Research 13: 105–25.

Margianti, E.S., B.J. Fraser, and J.M. Aldridge. 2001. Classroom environment and students’outcomes among university computing students in Indonesia. Paper presented at theAmerican Educational Research Association, Seattle, WA.

Marsh, H.W., K. Hau, and Z. Wen. 2004. In search of golden rules: Comment on hypothe-sis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneral-izing Hu and Bentler’s 1999 findings. Structural Equation Modeling: A MultidisciplinaryJournal 11, no. 3: 320–41.

Moos, R. 1991. Connection between school, work and family settings. In Educational envi-ronments: Evaluation antecedents and consequences, ed. B. Fraser and W. Walberg, 29–53. London: Pergamon.

Riah, H., and B.J. Fraser. 1998. The learning environment of high school chemistry classes.Paper presented at the annual meeting of the National Association for Research in Sci-ence Teaching, San Diego, CA.

Sireci, S.G., and G. Berberoġlu. 2000. Using bilingual respondents to evaluate translated-adapted items. Applied Measurement in Education 13: 229–48.

Soerjaningsih, W., B.J. Fraser, and J.M. Aldridge. 2001. Achievement, satisfaction and learn-ing environment among Indonesian computing students at the university level. Paper pre-sented at the Annual Meeting of the American Educational Research Association,Seattle, WA.

Telli, S., J. Çakıroğlu, and P. den Brok. 2006. Turkish secondary education students’ percep-tions of their classroom learning environment and their attitude towards biology. In Con-temporary approaches to research on learning environments, ed. D.L. Fisher and M.S.Khine, 517–42. Singapore: World Scientific.

Research in Science & Technological Education 59

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

Appendix 1. TROFLEI in Turkish and English

TROFLEI scale Turkish version English version

Öğrenciler ArasıUyum – StudentCohesiveness

Sınıfımdaki öğrenciler ilearkadaşlıklar kurarım

I make friends among students inthis class

Sınıfımdaki diğer öğrencileritanıyorum

I know other students in thisclass

Sınıfımdaki öğrencilere arkadaşçadavranırım

I am friendly to members of thisclass

Sınıfımdaki öğrenciler benimarkadaşlarımdır

Members of the class are myfriends

Sınıfımdaki diğer öğrenciler ileuyumlu çalışırım

I work well with other classmembers

Sınıfımda ders ile ilgili zorlukyaşayan diğer öğrencilere yardımcıolurum

I help other class members whoare having trouble with theirwork

Sınıfımdaki öğrenciler beni sever Students in this class like meSınıf arkadaşlarımdan yardımalırım

In this class, I get help fromother students

Öğretmen Desteği– TeacherSupport

Öğretmen benimle bire bir ilgilenir The teacher takes a personalinterest in me

Öğretmen bana yardımcı olmakiçin farklı yollar dener

The teacher goes out of his/herway to help me

Öğretmen benim duygularımıdikkate alır

The teacher considers myfeelings

Ders ile ilgili sorun yaşadığımdaöğretmen bana yardımcı olur

The teacher helps me when Ihave trouble with the work

Öğretmen benimle iletişimkurmaya çalışır

The teacher talks with me

Öğretmen benim problemlerim ileilgilenir

The teacher is interested in myproblems

Öğretmen derste bana yardımcıolmak için yanıma gelir

The teacher moves about theclass to talk with me

Öğretmenin sorduğu sorularkonuyu anlamama yardımcı olur

The teacher’s questions help meto understand

Katılım –

InvolvementDerste fikirleri tartışırım I discuss ideas in class

Sınıf tartışmalarında görüşlerimiaçıklarım

I give my opinions during classdiscussions

Öğretmen bana sorular sorar The teacher asks me questionsDüşünce ve önerilerim sınıftartışmalarında kullanılır

My ideas and suggestions areused during classroomdiscussions

Öğretmene sorular sorarım I ask the teacher questionsGörüş ve düşüncelerimi sınıfarkadaşlarıma açıklarım

I explain my ideas to otherstudents

Sınıf arkadaşlarım problemlerinnasıl çözümleceğini benimletartışırlar

Students discuss with me how togo about solving problems

Sorunları nasıl çözdüğümünaçıklanması istenir

I am asked to explain how Isolve problems

Araştırma – TaskOrientation

Görüşlerimi doğrulamak içinaraştırmalar yaparım

I carry out investigations to testmy ideas

(Continued)

60 A.G. Welch et al.

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

Appendix 1. (Continued).

TROFLEI scale Turkish version English version

Açıklamaların dayandığı kanıtlarhakkında düşünmem istenir

I am asked to think about theevidence for statements

Tartışmalar sonucu ortaya çıkansoruları cevaplamak içinaraştırmalar yaparım

I carry out investigations toanswer questions coming fromdiscussions

İfadelerin, şekillerin ve grafiklerinanlamlarını açıklayabilirim

I explain the meaning ofstatements, diagrams and graphs

Aklıma takılan soruların yanıtınıbulmak için araştırmalar yaparım

I carry out investigations toanswer questions that puzzle me

Öğretmenin sorularını yanıtlamakiçin araştırmalar yaparım

I carry out investigations toanswer the teacher’s questions

Soruların yanıtlarını araştırmayaparak bulurum

I find out answers to questionsby doing investigations

Problemleri kendi araştırmalarımile elde ettiğim bilgileri kullanarakçözerim

I solve problems by usinginformation obtained from myown investigations

Görev Bilinci –Investigation

Belirli bir çalışma düzeniniyakalamak benim için önemlidir

Getting a certain amount of workdone is important to me

Başlarken belirlediğim hedeflereulaşana kadar çalışırım

I do as much as I set out to do

Bu dersin amaçlarını biliyorum I know the goals for this classBu derse her zaman hazır olarakgelirim

I am ready to start this class ontime

Bu derste benden nelerinbeklendiğini biliyorum

I know what I am trying toaccomplish in this class

Bu dersi dikkatli bir şekilde takipederim

I pay attention during this class

Bu dersteki konuları veuygulamaları anlamaya çalışırım

I try to understand the work inthis class

Ne kadar çalışmam gerektiğinibiliyorum

I know how much work I haveto do

İşbirliği –Cooperation

Ödev yaparken sınıf arkadaşlarımile işbirliği yaparım

I cooperate with other studentswhen doing assignment work

Ödevleri yaparken kitap vekaynaklarımı sınıf arkadaşlarım ilepaylaşırım

I share my books and resourceswith other students when doingassignments

Bu derste yapılan grupçalışmalarında takım ruhu ortayaçıkar

When I work in groups in thisclass, there is teamwork

Bu derste sınıf arkadaşlarım ilebirlikte ortak projeler üzerindeçalışırım

I work with other students onprojects in this class

Bu derste sınıf arkadaşlarımdanöğrenirim

I learn from other students in thisclass

Bu derste sınıf arkadaşlarım ilebirlikte çalışırım

I work with other students in thisclass

Ders aktivitelerinde sınıfarkadaşlarım ile işbirliği yaparım

I cooperate with other studentson class activities

Dersin hedeflerini başarmak içinsınıf arkadaşlarım benimle işbirliğiyaparlar

Students work with me toachieve class goals

(Continued)

Research in Science & Technological Education 61

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

Appendix 1. (Continued).

TROFLEI scale Turkish version English version

Sınıf içiDemokrasi veEşitlik – Equity

Öğretmen benim sorularıma diğerarkadaşlarımın sorularına verdiğikadar önem verir

The teacher gives as muchattention to my questions as toother students’ questions

Öğretmenden diğer öğrencilerinaldığı kadar yardım alırım

I get the same amount of help fromthe teacher as do other students

Bu derste benimde sınıfarkadaşlarım kadar söz hakkımvardır

I have the same amount of say inthis class as other students

Bu derste bana sınıftaki diğeröğrencilere davranıldığı gibidavranılır

I am treated the same as otherstudents in this class

Öğretmen sınıf arkadaşlarımaverdiği desteğin aynısını bana daverir

I receive the same encouragementfrom the teacher as other studentsdo

Sınıf tartışmalarına katkıdabulunmak için bana da sınıfarkadaşlarım kadar fırsat verilir

I get the same opportunity tocontribute to class discussions asother students

Çalışmalarım arkadaşlarımınçalışmaları kadar övgü alır

My work receives as much praiseas other students’ work

Soruları yanıtlamak için sınıfarkadaşlarımla eşit fırsatım vardır

I get the same opportunity toanswer questions as otherstudents

Farklılaşma –

DifferentiationKendime ait bir çalışma hızımvardır

I work at my own speed

Benden hızlı çalışan öğrenciler birsonraki konuya geçerler

Students who work faster thanme move on to the next topic

Derste konuları seçme hakkımvardır

I am given a choice of topics

Bana verilen görevler sınıfarkadaşlarımınkinden farklıdır

I am set tasks that are differentfrom other students’ tasks

Bana yeteneklerime uygun ödevlerverilir

I am given work that suits myability

Benim kullandığım materyallerarkadaşlarımın kullandıklarındanfarklıdır

I use different materials fromthose used by other students

Arkadaşlarımdan farklıdeğerlendirme yöntemlerikullanırım

I use different assessmentmethods from other students

Arkadaşlarımın çalışmalarındanfarklı çalışmalar yaparım

I do work that is different fromother students’ work

BilgisayarKullanımı –Computer Usage

Ödevlerimi hazırlarken bilgisayarkullanırım

I use the computer to type myassignments

Ödevlerimi öğretmene e-posta ilegöndermek için bilgisayarkullanırım

I use the computer to emailassignments to my teacher

Öğretmene soru sormak içinbilgisayar kullanırım

I use the computer to ask theteacher questions

Ders hakkında bilgi toplamak içinbilgisayar kullanırım

I use the computer to find outinformation about the course

Öğretmen tarafından hazırlananders notlarını okumak içinbilgisayar kullanırım

I use the computer to read lessonnotes prepared by the teacher

(Continued)

62 A.G. Welch et al.

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012

Appendix 1. (Continued).

TROFLEI scale Turkish version English version

Çalışmalarımın nasıldeğerlendirileceğini öğrenmek içinbilgisayar kullanırım

I use the computer to find outinformation about how my workwill be assessed

Bilgisayarı sınıf arkadaşlarım ileinternet üzerinde tartışmalaryapmak için kullanırım

I use the computer to take part inonline discussions with otherstudents

İnternetten bilgi toplamak içinbilgisayar kullanırım

I use the computer to obtaininformation from the Internet

Ergen Kültürü –

Young AdultEthos

Bana genç bir yetişkin gibidavranılır

I am treated like a young adult

Bana sorumluluk verilir I am given responsibilityBenden kendi adıma düşünmembeklenir

I am expected to think for myself

Bana yetişkin gibi davranılır I am dealt with as a grown-upGüvenilir biri olarak kabul edilirim I am regarded as reliableOlgun olduğum düşünülür I am considered matureBana bağımsız olma şansıverilmiştir

I am given the opportunity to beindependent

Kendi öğrenme sureciminsorumluluğunu ustlenmem içinteşvik edilirim

I am encouraged to take controlof my own learning

Research in Science & Technological Education 63

Dow

nloa

ded

by [

Nor

th D

akot

a St

ate

Uni

vers

ity]

at 0

9:29

20

Mar

ch 2

012