E-Learning Pedagogy and School Leadership Practices  to  Improve  Hong  Kong  Students’  

Computer and Information Literacy:

Findings from ICILS 2013 and beyond

Nancy LAW, Johnny YUEN, Yeung LEE Centre for Information Technology in Education, University of Hong Kong

Copyright © Centre for Information Technology in Education First published 2015

Published by Centre for Information Technology in Education (CITE) Faculty of Education The University of Hong Kong

The book is published with open access at icils.cite.hku.hk

Open Access. This book is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited. All commercial rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law   of   the   Publisher’s   location, in its current version, and permission for commercial use must always be obtained from the Publisher. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The National Research Centre for the Hong Kong participation in ICILS 2013 would like to acknowledge the financial support provided by the Hong Kong Quality Education Fund (QEF).

E-Learning Pedagogy and School Leadership Practices to Improve Hong Kong Students’   Computer   and   Information   Literacy:   Findings   from   ICILS 2013 and beyond/ by Nancy Law, Johnny Yuen and Yeung Lee.

ISBN 978-988-18659-6-0

Cover design by Debbie Pang

i

Acknowledgements

Hong Kong’s participation in ICILS 2013 would not have been possible without the

funding support for the project. We are very grateful to the support given by all the

principals, teachers, ICT coordinators and students who participated in the pilot

and main studies. Without their participation, it would have been possible to

complete the study so successfully and smoothly.

We need to highlight the enormous help we received from our Honorary

Consultant, Mrs Grace Kwok, who worked tirelessly to help us with liaison with

school principals.

We are also very thankful for the generous support given by the Education Bureau

and her staff, especially for helping the project to ensure participation from schools.

We are also indebted to steering committee for their valuable advice over the

duration of the project.

Finally, we would also like to extend our hearty thanks for all the colleagues

involved in various stages in this project. In particular, we would like to

acknowledge the contribution from a number of colleagues and students: Dr. Zhan

Wang for her contribution to the analysis of the data, Dr. Ling Li for her copy-

editing of the English version of this book and Mr Leming Liang for copy-editing

and formatting the Chinese version. We are also grateful to Mrs Liliana Farias for

her careful formatting of the English version.

ii

Project Team

Project Leader

Prof. Nancy LAW

Deputy Director, CITE, Faculty of Education, The University of Hong Kong

Research Team Members

Dr. Y. LEE

Assistant Director, CITE, Faculty of Education, The University of Hong Kong

Dr. Johnny YUEN

Research Officer, CITE, Faculty of Education, The University of Hong Kong (2013-

2015)

Dr. Emily OON

Research Officer, CITE, Faculty of Education, The University of Hong Kong (2011-

2013)

Ms. Ada TSE

Research Assistant, CITE, Faculty of Education, The University of Hong Kong

Mrs. Grace Kwok

Honorary Consultant: School Liaison

Test Administrators

AU Lok Yee AU Suet Yee CHAN Ka Lai

CHAN Ki Yan CHENG Kwan Yee CHEUNG Sau Ping

CHIANG Lai Hang CHOW Yan FUNG Hiu Tung

FUNG Lik Yan KAN Wai Kit LAM Wai Sum

LEE Ying Yin LIU Sin Yi NGAI Yik Long

WONG Chi Lai YAU Hiu Ying YUNG Man Yi

iii

Steering Committee Mr. Danny CHENG

Ex- Chairman, The Hong Kong Association for Computer Education

Mr. K. W. LAM

Ex- Chairman, Hong Kong Direct Subsidy Scheme Schools Council

Mr. Y.T. LAU (10/2014 - 09/2015)

CCDO, Information Technology in Educational Section, Education Bureau

Mr. S. T. LEUNG

Chairman, Hong Kong Aided Primary School Heads Association

Mr. A. C. LIU

Ex- Chairman, Hong Kong Subsidized Secondary Schools Council

Mrs. K. Y. LIU NG

Chairperson, Union of Government Primary School Headmasters and

Headmistresses

Ms. M. M. NG

Ex- CEO, HKEdCity

Mr. S. K. NG

Ex- Vice-Chairman, Hong Kong Association of Heads of Secondary Schools

Mr. M. SHE (08/2011 - 10/2014)

Ex- CCDO, Information Technology in Educational Section, Education Bureau

Mr. George TAM

Ex- Chairman, Hong Kong Grant School Council

Mr. Albert WONG

Chairman, Association of IT Leaders in Education (AiTLE)

iv

v

Table of Contents

Acknowledgements i

Project Team ii

Steering Committee iii

Table of Contents v

List of the Tables ix

List of the Figures xiii

Executive Summary xvii

Chapter 1: Computer & Information Literacy and its Assessment 1

1.1 Computer and Information Literacy: a Brief History 2 1.2 Computer and Information Literacy in the Hong Kong School

Curriculum 4

1.3 Assessing Computer and Information Literacy 7

1.3.1 European Computer Driving Licence 8 1.3.2 Computer-Based Assessments in PISA 9

1.4 Definition of CIL in ICILS 13 1.5 Test Administration and Module Design in ICILS 15 1.6 Summary 20 Chapter  2:  Hong  Kong  Students’  Performance  in  ICILS 21

2.1 Sampling Method and Participation Rates 21 2.2 Computation  of  CIL  Scores  and  Countries’  Performance 22 2.3 CIL Proficiency Levels 25 2.4 Hong  Kong   Students’   CIL   Proficiency   Levels   in   an   International  

Context 27

2.5 Students’  Performance  across  CIL Aspects 30

2.5.1 Knowing about and understanding computer use (Aspect 1.1)

31

2.5.2 Accessing and evaluating information 32 2.5.3 Managing information 32 2.5.4 Transforming information 32 2.5.5 Creating information 35

vi

2.5.6 Sharing information 36

2.5.7 Using information safely and securely 37

2.5.8 Students’  performance  across  CIL  aspects  and  proficiency  levels

40

2.6 Students’  CIL  proficiency  trajectories 42

2.6.1 Hong  Kong  students’  improvements  in  competency  profile

as they move to higher CIL proficiency levels

44

2.6.2 Australian   and   Korean   students’   improvements   in  competency profile as they move to higher CIL proficiency

levels

45

2.7 Summary 47

Chapter  3:  Influence  of  Students’  Background  and  ICT Use Experience

on their CIL

49

3.1 Contextual Factors Derived from the Student Survey 50

3.2 Influence  of  Students’  Personal  and  Family  Background 51

3.2.1 Gender and CIL achievement 52

3.2.2 Educational aspirations and CIL achievement 53

3.2.3 Immigrant status and CIL achievement 54

3.2.4 Language use at home and CIL achievement 55

3.2.5 Socioeconomic background and CIL achievement 56

3.2.6 Home ICT resources and CIL achievement 57

3.3 Influence  of  Students’  ICT  Self-efficacy and Interest 58

3.3.1 Self-efficacy in basic ICT skills and CIL achievement 58

3.3.2 Self-efficacy in advanced ICT skills and CIL achievement 60

3.3.3 Interest and enjoyment in using ICT and CIL achievement 61

3.4 Influence of Students’  ICT  Use  Experience  at  Home  and  in  School 64

3.4.1 Computer experience and CIL achievement 64

3.4.2 Use of computers for and at school 67

3.4.3 Use of computers outside school 74

3.5 Students’  Contextual  Factors  and  Their  CIL  Achievement 79

3.5.1 Contextual factors and overall CIL score 79

3.5.2 Contextual factors and the seven CIL aspect scores 82

3.6 Summary 85

vii

Chapter  4:  How  Do  Schools  Influence  Students’  CIL  Achievement? 87

4.1 ICT infrastructure and resources in schools 88 4.1.1 Digital learning resources 88 4.1.2 Computer resources for teaching and/or learning 91 4.1.3 Student: Computer ratios 92 4.1.4 Summary 93

4.2 School Policies and Practices Regarding ICT Use 94

4.2.1 Principals’  views  on  educational  purposes  of  ICT  use 94 4.2.2 Principals’  expectations  of  teachers’  knowledge  and  skills  in  

professional use of ICT 95

4.2.3 The  extent  to  which  principals  monitored  teachers’  ICT  use  to achieve different learning outcomes

96

4.2.4 The extent to which principals took main responsibility for ICT management and implementation

100

4.2.5 The extent to which schools had measures regarding ICT access and use

101

4.2.6 The extent to which teachers participated in different forms of professional development as reported by principals

103

4.2.7 Principal’s  priorities  for  facilitating  use  of  ICT 105 4.2.8 Obstacles   that   hinder   school’s   capacity   to   realize   its   e-

Learning goals 107

4.3 Teacher’s  ICT-using Pedagogy 108

4.3.1 Teacher confidence in using ICT 110 4.3.2 Teachers’  reported  use  of  ICT  tools  in  teaching 112 4.3.3 Teachers’  reported  student  use  of  ICT  in  different  learning  

tasks 114

4.3.4 Teachers’  reported  use  of  ICT  for  various  types  of teaching and learning activities

116

4.3.5 Teachers’   emphasis   on   developing   students’   information  literacy

118

4.4 School  Level  Factors  and  Hong  Kong  Student’s  CIL  Achievement 120 4.4.1 School level factors and overall CIL score 121 4.4.2 School level factors and the seven CIL aspect scores 125 4.5 Summary 126

Chapter  5:  Learning  &  Assessment  Designs  to  foster  students’  CIL 129

viii

5.1 Principles of Learning Design for CIL Development 129 5.2 Principles of Assessment Design 130 5.3 Learning Designs Targeting Specific CIL Aspects 132 5.3.1 Learning designs to foster skills for accessing and

evaluating information 132

5.3.2 Learning designs to foster skills for managing information 138 5.3.3 Learning designs to foster skills for transforming

information 141

5.3.4 Designs to foster skills for sharing information 142

5.4 An Extended Learning Design to Foster Learning in Multiple CIL Aspects

146

5.5 Summary 154

References 155

ix

List of Tables Table 1.1 The key standards areas specified in the 1998 and

2007 NETS 3

Table 1.2 The Computer and Information Literacy Framework adopted in ICILS

14

Table 1.3 A comparison of the IL assessment framework adopted by the ILPA Study in Hong Kong (Law et al. 2008) and that used in ICILS

15

Table 1.4 Overview of the short tasks in the After-School Exercise module

17

Table 1.5 Summary of the performance expectation and associated CIL aspect for each of the score point criteria in the After-School Exercise long task module

19

Table 2.1 Allocation of module score points to the ICILS assessment framework

23

Table 2.2 Country averages for CIL, years of schooling, average age, ICT Index, student–computer ratios and percentile graph

24

Table 2.3 Descriptions  of  students’  typical  performance  capabilities at the different levels of the CIL scale

26

Table 2.4 Examples of ICILS test items mapped to the four CIL proficiency levels based on their item difficulty

28

Table 2.5 The percentages correct (S.D.) and relative difficulty for the seven CIL aspects for Hong Kong, Australia and Korea

30

Table 2.6 Sample responses from Hong Kong students to the three tasks related to the phishing email

38

Table 2.7 Samples  of  Hong  Kong  students’  responses  to  the  short task on problems that may arise by making one’s  email  address  public

40

Table 2.8 Percentage of students who have correctly answered each short task in the After-school Exercise module (mapped to CIL assessment aspect and level)

41

Table 2.9 Percentage of students who have achieved partial or full score for each assessment criterion (mapped to CIL level) in the poster design large-task

42

Table 3.1 List of key context variables* derived from the student questionnaire

51

Table 3.2 Gender differences in CIL 52

x

Table 3.3 The percentages of students at each level of educational aspiration and their respective CIL scores

53

Table 3.4 The percentages of students at each level of parental education reached and their respective CIL scores

56

Table 3.5 The percentages of students at each level of home literacy and their respective CIL scores

56

Table 3.6 The percentages of students with different numbers of computers at home and their respective CIL scores

57

Table 3.7 The percentages of students confident in performing each basic ICT skills task, S_BASEFF and correlation with CIL scores

59

Table 3.8 The percentages of students confident in performing each basic ICT skills task, S_ADVEFF and correlation with CIL scores

62

Table 3.9 Percentages of students agreeing with statements about computers, S_INTRST and correlation with CIL scores

63

Table 3.10 Percentages of students with different years of experience with computers

65

Table 3.11 Percentages of students with frequent computer use (i.e. at least once a week) at home, school and other places

67

Table 3.12 Percentages of students using computers in most lessons or almost every lesson in different learning areas, S_UELRN and association with CIL score

68

Table 3.13 Percentages of students using computers for study purposes at least once a month, S_USESTD and association with CIL score

71

Table 3.14 Percentages of students reported having learnt CIL related tasks at school, S_TSKLRN and the association with CIL score

73

Table 3.15 Percentages of students using work-oriented applications outside school at least once a week, S_USEAPP and the association with CIL score

75

Table 3.16 Percentages of students using the Internet outside school at least once a week for exchange of information, S_USEINF and the association with CIL score

77

Table 3.17 Percentages of students using the Internet outside school at least once a week for social

78

xi

communication, S_USECOM and the association with CIL score

Table 3.18 Percentages of students using the Internet outside school at least once a week for recreation, S_USEREC and the association with CIL score

80

Table 3.19 Multilevel  model  results  for  students’  CIL  scores  using student context variables as level 1 predictors

81

Table 3.20 Multilevel model results for each of the seven standardized CIL aspect scores using student context variables as level 1 predictors

84

Table 3.21 List of key student context scale variables and their correlation with CIL scores for Hong Kong

85

Table 4.1 Percentages of students studying at schools with available internet-related teaching and learning resources

89

Table 4.2 Percentages of students studying at schools with available software resources for teaching and/or learning

90

Table 4.3 Percentages of students at schools with computer resources for teaching and/or learning

91

Table 4.4 Percentages of students at schools with school computers at different locations

93

Table 4.5 Percentages of students at schools where the principals consider ICT use as very important for achieving different educational outcomes

95

Table 4.6 Percentages of students at schools where the principals expect and require teachers to have different technological pedagogical knowledge and communication skills

97

Table 4.7 Percentages of students at schools where the principals  use  various  means  to  monitor  teachers’  ICT  use  to  develop  students’  advanced  ICT  skills

99

Table 4.8 Percentages of students at schools where the principals use various  means  to  monitor  teachers’  ICT use

99

Table 4.9 Percentages of students at schools where the principals took main responsibilities for different aspects of ICT management and implementation

100

Table 4.10 Percentages of students at schools where the principals took main responsibilities for different aspects of ICT management

102

xii

Table 4.11 Percentages of students at schools where teachers

participate in different ICT- related professional

development as reported by their principals

104

Table 4.12 Percentages of students at schools where principals

indicate medium or high priority to ways of

facilitating use of ICT in teaching and learning

106

Table 4.13 Percentages of Hong Kong principals who indicate

“somewhat”  or  “a  lot”  of  hindrance  cause  by issues

listed  to  their  school’s  capacity  to  realize  e-Learning

capacity

109

Table 4.14 Percentages of teachers expressing confidence in

doing different computer tasks

111

Table 4.15 Percentages of teachers using ICT tools in most,

almost every, and every lessons

113

Table 416 Percentages of teachers who indicate students often

using ICT for teaching activities in classrooms

115

Table 4.17 Percentages of teachers often using ICT for teaching

practices in classrooms

117

Table 4.18 Percentages of teachers put strong or some

emphasis  to  develop  students’  ICT-based

119

Table 4.19a Table 4.19a List of key school-level variables*

derived from the principal

121

Table 4.19b List of key school-level variables* derived from the

teacher questionnaire

122

Table 4.20 Multilevel  model  res0.0ults  for  students’  CIL  scores  using school-level variables as level 2 predictors (z-

scores)

124

Table 4.21 Multilevel model results for each of the seven

standardized CIL aspect scores using school level

variables as level 2 predictors

127

Table 5.1 Assessment  rubrics  for  evaluating  “creating  information”

131

Table 5.2 An  assessment  checklist  for  “creating  information”  in a poster creation task

132

Table 5.3 An  assessment  rubric  for  students’  work  on  evaluating information

135

Table 5.4 A self-evaluation checklist and associated

assessment rubric for the website evaluation task

137

Table 5.5 Assessment criteria for the poster and route map 141

Table 5.6 An assessment rubric for peer review type of forum

postings

145

xiii

List of Figures Figure 1.1 Logo for European Computer Driving Licence

Foundation (ECDL)

1

Figure 1.2 National Education Technology Standards. 8

Figure 1.3 The seven pillars of information literacy according

to SCONUL.

4

Figure 1.4 Relationship of IL to lifelong learning 4

Figure 1.5 Modules included in the Information Technology

Learning Targets guideline

5

Figure 1.6 The nine generic skills specified in the Reform

Proposal

5

Figure 1.7 Consultation documents published by the

Education Commission in 2000 with a focus on

preparing students for lifelong and lifewide

learning

6

Figure 1.8 The Information Literacy Framework released by

the EMB in 2005 and the graphical representation

for information literacy used in the document

6

Figure 1.9 Graphical representations of the Information

Literacy and Scientific Inquiry frameworks used by

the ILTOOLS team

7

Figure 1.10 The Australian ICT Literacy Assessment

framework

8

Figure 1.11 An item in the Primary 5 and Secondary 2 ILPA

generic  test  that  assesses  the  ‘integrate’  dimension

10

Figure 1.15 An item in the Primary 5 ILPA mathematics test

that  assesses  the  ‘integrate’  dimension

11

Figure 1.13 Screen dumps for a task in the Secondary 2 ILPA

Science  test  that  assesses  the  ‘integrate’  dimension

12

Figure 1.14 A sample screen in an ICILS test module showing

the functions for different sections of the screen

16

Figure 1.15 The poster design environment, information

resources, instructions and assessment criteria

provided to students for working on the large task

18

Figure 2.1 Percentages of Hong Kong, Australia and Korea

students performing at each CIL proficiency level

29

Figure 2.2 The  three  tasks  that  test  students’  ability  to  navigate to a designated webpage using different

forms of instruction

31

Figure 2.3 A poster designed by a Hong Kong student 32

xiv

Figure 2.4 Sample poster designed by Hong Kong student 32 Figure 2.5 Screenshot of the short task in After-school

Exercise that tests students’  ability  to  change  the  sharing setting of a web document

33

Figure 2.6 Sample poster designed by a Hong Kong student 34 Figure 2.7 Sample poster designed by a Hong Kong student 35 Figure 2.8 The  short  task  that  assessed  students’  awareness  of  

risks in making personal information public 37

Figure 2.9 The phishing email with three suspicious elements, A, B and C

39

Figure 2.10 A radar diagram of the mean percentages correct per CIL aspect

43

Figure 2.11 A radar diagram showing the mean percentages correct per CIL aspect for Hong Kong students at each of the five CIL proficiency levels

44

Figure 2.12 A radar diagram showing the mean percentages correct per CIL aspect for Australian students at each of the five CIL proficiency levels

46

Figure 2.13 A radar diagram showing the mean percentages correct per CIL aspect for Korean students at each of the five CIL proficiency levels

47

Figure 3.1 Contextual  factors  influencing  students’  CIL  outcomes

50

Figure 5.1 Sample*  of  students’  submitted work on ancient calculating tools

132

Figure 5.2 Figure 5.2 Sample of the comic strips created by the students

134

Figure 5.3 A  P.6  student’s  work  on  evaluating  information  from news reports

135

Figure 5.4 A  Secondary  3  student’s  work  on  evaluating information from websites

136

Figure 5.5 Sample  of  P.  2  students’  work  on  managing  information

138

Figure 5.6 Samples  of  P.  5  students’  template  for  recording  experimental data

139

Figure 5.7 Data collection template resubmitted by Group 3 140 Figure 5.8 A sample poster and field trip route on Google

map produced by a group of P.5 students 143

Figure 5.9 Google calendar used by some students to plan their work schedule

144

xv

Figure 5.10 An example of a Learning Management System customized to support shared reading and peer commenting of essays

144

Figure 5.11 A discussion forum where students commented on the lyrics written by their classmates

145

Figure 5.12 Postings of information on the discussion forum in

Stage 1, demonstrating students’  ability  to  use  keywords to access information, use the forum to manage and share information

147

Figure 5.13 The target shop, route plan and interview questions prepared by one group of students in Stage 2, demonstrating their abilities to access, manage and transform information

148

Figure 5.14 In Stage 3, students conducted the interview during the field trip, then posted the collected information and their own reflections online after the visit

150

Figure 5.15 Artefacts created by students in Stage Four: The group mind map and a 3D paper model created by a student who completed his essay writing early. These  artefacts  demonstrate  students’  ability  to  transform and create information

151

Figure 5.16 An essay written by one of the students, demonstrating his ability to transform and create information

152

Figure 5.17 The rubric for self- and peer- assessment of the essays, and the peer feedback given by students to their peers online

153

xvi

xvii

Executive Summary Background

As the title of this book indicates, the focus of this publication is to help teachers,

principals, education policy makers, teacher educators and members of the

community  concerned  about  student  learning  to  understand  Hong  Kong  students’  levels of Computer Information Literacy (CIL) achievement in comparison with

their international peers, and what e-learning pedagogy and e-leadership practices

in schools will help to foster students’  ability  to  make  use  of  ICT tools productively

for lifelong learning in the 21st century. The core of this book is an in-depth analysis

of the Hong Kong results from the International Computer and Information

Literacy Study 20131 (ICILS2013), drawing on comparisons with the international,

Australian, and Korean data. Findings from this analysis will help us to understand

the strengths  and  weaknesses  of  Hong  Kong  students’ CIL achievement, as well as

the contextual factors (personal, family, teacher and school leadership factors) that

influence them. This book also provides curriculum exemplars to illustrate the key

characteristics of pedagogical and assessment designs that are conducive to

enhancing  students’  CIL, drawing on past and current CITE projects.

The International Computer and Information Literacy Study 2013 2

(ICILS2013) is the first large-scale   international   comparative   study   on   students’  ability to make use of computer and information technology to conduct inquiry,

create, communicate and use information safely at home, school and different

social and workplace contexts. Including Hong Kong, a total of 21 countries and

education systems participated in this study, which was conducted under the

auspices of the International Association for the Evaluation of Educational

Achievement (IEA). The Hong Kong component of the ICILS 2013 study was

conducted by the Centre for Information Technology in Education (CITE) of The

University of Hong Kong, funded by the Quality Education Fund (QEF). The actual

data collection took place between March and July of 2013.

Computer and Information Literacy—concept and test design

The assessment framework for ICILS 2013 (Fraillon et al., 2013) comprises two

strands of abilities. The first strand, collecting and managing information, can be

further differentiated into three aspects: knowing about and understanding

computer use, accessing and evaluating information, and managing information.

1 Hong Kong ICILS 2013 Study website: http://icils.cite.hku.hk/ 2 ICILS 2013 International Study website: http://www.iea.nl/icils_2013.html

xviii

The second strand, producing and exchanging information, encompasses four aspects: transforming information, creating information, sharing information, and using information safely and securely.

The performance assessment was conducted on computers located in the computer labs of the sampled schools. The student test consisted of questions and tasks presented in four 30-minutes modules. Each participating student was randomly assigned to complete two of the test modules. Each module comprises multiple-choice items and constructed responses pertaining to authentic tasks, designed according to the CIL assessment framework.

Similar to all IEA studies of student achievement, the ICILS2013 CIL scale has an  average  score  set  to  500  and  standard  deviation  to  100.  In  addition,  students’  CIL performance are categorized into five proficiency levels in descending order: level 4, level 3, level 2, level 1, and below level 1. Table 2 presents detail descriptions of the five CIL proficiency levels.

ICILS 2013 Study design and HK participation statistics

ICILS 2013 requires the following from all participating systems: x School sample: A random sample of at least 150 secondary schools from

the target population of schools that offer grade 8 classes in the 2012-13 academic year.

x School questionnaire data collection: Online questionnaire to be completed by the school principal, the ICT coordinator, and 15 teachers who teach grade 8 classes in the 2012-13 academic year (~20-30 minutes).

x Student sample: 20-25 students randomly sampled from all grade 8 students in each participating school.

x Student data collection: Computerized student CIL test (60 minutes) and student questionnaire (20 minutes).

In Hong Kong data, 118 secondary schools participated in the ICILS 2013 study. A total of 2089 Secondary 2 students, 1338 Secondary 2 teachers, 115 principals and 105 ICT coordinators from the sampled schools took part in the study. Overall participation rates, after weighting and replacement, are: students 68.6%, teachers 58.3%, schools 70.8%. The Hong Kong set of data is considered as belonging to category 2, and does not meet the IEA standards for statistical comparisons across countries (category 1), which require overall participation rates after weighting and replacement to be at least 75%.

Students’  CIL  achievement Hong Kong Secondary 2 students’  overall  average  CIL  score  is  509,  slightly  above  the ICILS 2013 average of 500. However, this is lower than all the economically developed participating educational systems.

xix

HK’s   ICT   Development   Index   is   ranked   fifth   among   the   21   participating  systems, but unfortunately, this strength in our ICT infrastructure contrasts with our  students’  relatively low CIL scores.

Analysis of HK  student’s  CIL performance across the items in the two CIL strands indicates a relatively weaker performance in strand 2 (producing and exchanging information). Further in-depth analysis of each of the seven aspects of CIL show that Hong Kong students performed better in CIL tasks on knowing about and understanding computer use, as well as using information safely and securely. Their performance are poorest if they had to process the information collected, that is, evaluating information, transforming information, sharing information, as well as managing information.

Of the five proficiency levels of CIL competence defined by IEA on the basis of  the  students’  performance,  a  student  need  to  be  able  to  perform  at  least  at  level 3 in order to be able to cope with the CIL needs for everyday needs. However, this amounts to only 26% for Hong Kong, compared to 34% for Australia and 35% for Korea. At the same time 38% of Hong Kong students achieved level 1 or below, which is relatively high among all economically developed participating systems. A total of 15% of HK students were assessed to be below level 1 CIL proficiency, which is very poor in comparison with Australia (5%) and Korea (9%). The standard  error  of  HK  students’  mean  CIL score is the second highest among all participating systems, indicating large variations in CIL among HK students.

By comparing the comparative improvements in mean percentage scores for each of the seven aspects for students from below level 1 (mean CIL score below 407) to level 4 (mean CIL score above 661). We find that Australian and Korean students generally showed balanced advancement in all seven aspects as we examine the means CIL aspect score from lower to higher proficiency levels. On the other hand, Hong Kong students were not able to show similar magnitude of advancement in all seven CIL aspects. In particular, no significant advancement could be found between level 3 and level 4 students in the two CIL aspects that Hong Kong students were poorest at: managing information and sharing information. Clearly, helping students to improve on their performance in these CIL aspects is a high priority.

HK  students’  personal  and  home  background,  and  how  these  relate to their CIL performance

SES

Similar to most other countries participating in the study, the CIL score of HK Secondary 2 female students is significantly higher than their male counterparts. Not surprisingly, there is a statistically significant positive correlation between HK students’  family  social  economic  status  (SES)  and their CIL scores.

xx

However the effect of SES on CIL achievement for Hong Kong students is not as high as in other education systems.

ICT access

Access to at least one computer at home (including desktop, notebook, or tablet computer) by surveyed students is high, at 98%. The mean CIL score of students with no computer at home is significantly lower, at CIL proficiency level 1. Students with at least one computer at home have mean CIL score at CIL proficiency level 2. The influence of having more than one computer at home on CIL performance is relatively minor. It is also observed that only less than 1% of the participating students reported not having Internet access at home, suggesting that some students could only access the Internet at home using smartphones.

Student level contextual factors and CIL achievement

We investigated whether  and  to  what  extent  Hong  Kong  students’  personal and family context affect their achievement in CIL. Correlational analyses show that students’  CIL  scores correlate with all student level contextual factors. However, many of these context factors are themselves highly correlated. Multilevel analyses using student level factors show  that  only  student’s  self-efficacy in basic ICT skills has the largest positive significant   influence   on   student’s   CIL   achievement.   In  addition,  students’  educational  aspirations  and their reports on opportunities to learn CIL-related tasks at school also significant positive coefficients. On the contrary,   students’   self-efficacy in advanced ICT skills and reported use of ICT during lessons at school had significant negative coefficients in predicting students’   CIL   achievement.   Further   multilevel   modeling of the relationship between student’s  context  variables  and their performance in each of the seven CIL aspects show similar results. These findings indicate that CIL proficiency is different from advanced ICT skills, and that the use of ICT during lessons in Hong Kong schools, even when it happened, was not conducive to the development of students’  CIL  proficiency.  Hence  it  is  important  to  explore  the  school  level  factors  that  contribute  positively  to  students’  CIL.  

School factors and their influence  students’ CIL achievement

Students’  e-learning opportunities and experiences are very much determined by the ICT infrastructure and digital learning resources available, as well as the nature and intensity of ICT use in pedagogical practices in their schools. In this study, three types of school level factors were explored: the ICT infrastructure and resources available, school policies and practices regarding ICT use and  teachers’  ICT-using pedagogy.

xxi

ICT infrastructure and resources available in schools

HK students had relatively good access to computers and the Internet at school for

instructional purposes: 100% had a computer lab and 84% had computers available

in most classrooms in their schools. There was also no relative lack of digital

learning resources for students in Hong Kong.

However, computers that students could access for e-learning at school, e.g.

through class sets of computers that can be moved between classrooms or on

computers brought by the student to class, were relatively low. In terms of network

infrastructures to support learning, HK was comparatively lower on internet-based

applications for collaborative work, and access to a learning management system

(65%) as compared with Korea (94%).

School policies and e-learning leadership factors in schools

School principals play an important part in determining the priorities and strategic

directions  of   the   school.  HK  principals’   top   three  priorities   related   to  e-learning

were establishing or enhancing an online learning support platform (87%),

increasing the bandwidth of Internet access for computers (84%), and increasing

the range of digital learning resources (83%), whereas their mean priorities for

pedagogical use of ICT in teaching and learning were lower than those reported in

other participating systems. In contrast, Australian   and   Korean   principals’   top  priorities  reflect  their  concern  about  teacher’s  pedagogical  use  of  ICT  as  well  as  the  range of e-Learning resources at school. Providing for participation in professional development on pedagogical use of ICT was considered  by  97%  of  Australian  students’  principals  and  89%  of  Korea  students’  principals  as  of  at   least  medium  priority.  Among  Korean   students’   principals,   the   top   three   priorities  were:   increasing the professional learning resources for teachers in the use of ICT (96%), establishing or enhancing online learning support platforms (94%), and providing teachers with incentives to integrate ICT use in their teaching (90%).

In terms of the educational purposes of ICT use in learning, HK principals

gave the highest priority to the three skill-oriented outcomes (all >80%): (1) basics

skills in using the office suite of applications and email, (2) proficiency in accessing

and using information, and (3) safe and appropriate use of ICT. Goals related to

improving   students’   general   learning   outcomes   and   fostering   students’  responsibility for their own learning were considered very important by only 64%

and 65%, respectively. This contrasts strongly with the Australian principals (93%

and 85%) and Korean principals (79% and 78%). Only 38% of Hong Kong students

attended schools whose principals consider ICT use to be very important for

developing   students’   collaborative   and   organizational skills, whereas the

international mean was 53%, and the Australian and Korean means were even

higher, at 73% and 68%, respectively.

xxii

It   is   thus  evident   that  Hong  Kong  principals’  views on the role of ICT for

learning and teaching were still focused on traditional outcomes, and gave much

lower priority to ICT use in fostering 21st century skills.

HK  principals   have  moderate   to   low   expectations   of   teachers’   ICT-related

knowledge and skills, except on their ability to use ICT to communicate with other

staff, at 86%. The highest expectation  was  on  teachers’  ability  to  integrate ICT into

teaching and learning, at 68%. Only 57% expected their teachers to be able to

collaborate with other teachers via ICT.

The lowest expectations were related to the use of ICT for assessment and

monitoring of students, neither of which had a percentage higher than 30%. In

particular, the percentage of students whose principals expected teachers to be able

to use ICT to develop authentic (real-life) assignments for students was only 16%.

This is very disappointing as bringing authentic contexts into the classroom is one

of the potential strengths that ICT use could offer.

Not  all  principals  would  monitor  teachers’  ICT  use  in  teaching.  In  Hong  Kong  and   internationally,   the  most  popular  means  principals  use   to  monitor   teachers’  ICT use in teachers was classroom observations. Another means of monitoring was

through teacher self-reflection.

Internationally, the aspect of ICT implementation that had the highest

proportion of principals taking key responsibility for was implementing ICT-based

approaches in administration, with a mean of 81%. Compared to the international

and Australian means, the percentages of Hong Kong principals taking main

responsibility for the various aspects of ICT management were relatively low,

except for the implementation of ICT-based approaches in administration. In

general, principals were least likely to take main responsibility for ICT

maintenance issues. It is interesting to note that none of the respective percentages

for Korean principals were higher than 25%, indicating that for Korean schools,

most of the ICT-specific management and implementation responsibilities were

devolved to other staff members.

In terms of having measures regarding ICT access and use by students in

school, HK principals were most concerned about ensuring that students would

not access unauthorized or inappropriate sites, and that they would honour

intellectual property rights—100% reported having measures to ensure these

conditions. Internationally, these are also the aspects that a vast majority of

students’   principals reported having implemented relevant measures. HK

principals were least concerned about restricting the total number of hours

students  were  allowed   to   sit   in   front  of  a   computer,  with  only  33%  of   students’  principals reported having measures in place regarding this. An even lower

percentage   of   Australian   students’   principals   reported   having   this   type   of  measures in place (18%), whereas the respective percentage for Korea was much

higher, at 64%.

xxiii

Based   on   the   principals’   reporting,   the most popular form of ICT-related

professional development activity was courses provided by the school, followed

by informal discussions within groups of teachers as well as discussions on ICT use

as a regular item embedded into staff meetings. In both Hong Kong and Korea, the

most popular professional development activities were courses provided by the

school and observing colleagues using ICT in their teaching. However, the levels

of participation reported by Hong Kong principals were very much lower than

even the international average. This may be one of the reasons for the low levels of

ICT adoption in teaching and student learning reported by teachers in Hong Kong.

The top issues that HK principals indicated as hindrances to their school were

pedagogy related, for example: insufficient time for teachers to implement e-Learning

(59%) is the number one hindrance as perceived by HK principals, followed by

insufficient budget for the needs of ICT implementation (e.g. LMS) (46%) as well as

insufficient qualified technical personnel to support the use of ICT (35%). More than one

third (34%) of HK principals indicated that pressure to score highly on

standardized tests was an obstacle. The data further suggest that lack of hardware

or general ICT skills among teachers were not perceived as big obstacles to e-

learning implementation in their schools by HK principals.

Teacher’s  ICT-using Pedagogy

Teachers play the biggest role in determining the learning experiences of students.

One   factor   that   potentially   impact   on   teachers’   use of ICT in their pedagogical

practices is their confidence in the use of ICT. A large majority of HK teachers

surveyed knew how to perform general and some advanced ICT tasks, with

percentages higher than the international average. This stands in stark contrasts to

the  teachers’  self-reported competence in ICT use for pedagogically related tasks,

namely   monitoring   students’   progress   (52%),   and   assessing   students’   learning  (58%), which were much lower than the corresponding percentages reported by

Australian (respectively 86% and 83%) and Korean (respectively 62% and 82%)

teachers.

HK teachers reported very low usage of ICT by the Secondary 2 students for

learning activities in the classroom, with most of the learning activities surveyed at

only a single digit percentage. The highest percentages recorded were for working

on extended projects (i.e. over several weeks) (12%), and searching for information

on a topic using outside resources (11%). Hong  Kong  students’ ICT usages in all

types of learning activities were either equal to, or lower than the corresponding

international averages. On the other hand, the corresponding percentages reported

by Australian teachers were generally higher than the international mean, with

often use of ICT reported by more than 30% in three activities: working on

extended projects, submitting completed work for assessment and searching for

information on a topic using outside resources.

xxiv

HK teachers reported a somewhat higher-level use of ICT in their teaching activities as opposed to ICT use by students for learning. However, only one aspect of ICT use by Hong Kong teachers in the classroom is higher than the ICILS international average, namely presenting information through direct class instruction (38%). All other types of ICT usage in teaching activities by Hong Kong teachers were lower than the ICILS international averages by 1% to 8%. The greatest differences between the Hong Kong and ICILS average (8%) were found in   two   kinds   of   teacher   use   of   ICT   to   support   students’   collaborative inquiry activities: collaboration among students (8%) and supporting inquiry learning (6%), which are often considered as important pedagogical activities to foster 21st century learning outcomes.

HK teachers also reported much lower emphasis on developing  their  students’  ICT-based capabilities compared to the international average. The largest gaps are found  in  teacher’s  emphasis  on  developing  student’s  ICT-based capabilities for (1) exploring a range of digital resources when searching for information (33% for Hong Kong compared to the international mean of 53%); (2) evaluating the relevance of digital information (36% compared to 52%); and (3) evaluating the credibility of digital information (also 36% compared to 52%). Only two out of the list of ICT-based capabilities were reported as emphasized by more than 50% of Hong Kong teachers, namely, accessing information efficiently (53%) and using computer software to construct digital work products (51%). In contrast, more than 50% of Australian teachers reported giving emphasis to the development of 11 out of the 12 ICT-based capabilities surveyed, and more than 50% of Korean teachers reported giving emphasis to helping their students to develop ICT-based capabilities in 9 out of the 12 surveyed.

HK teachers’  reported  usage  of  ICT  for  collaboration  among  fellow  teachers  is in general lower than the international average. Specifically, only 39% of HK Secondary 2 teachers have collaborated with peers to develop lessons involving use of ICT, compared to the international average of 58%.

Status of e-learning related school level factors in Hong Kong

In summary, the Study reveal that HK principals in general gave relatively lower priority to pedagogical use of ICT, particularly with respect to the use of ICT for developing  students’   21st century skills such as collaborative and organizational skills.   HK   teachers’   participation   in   ICT-related professional development activities was very much lower than the corresponding international average. Their reported use of ICT for teaching and learning were also very low. For most of the student learning activities surveyed, less than 10% of teachers reported that their Secondary 2 students often use ICT for those activities. HK teachers also reported low  emphasis  on  developing  their  students’  CIL  capabilities.  

xxv

How  did  school  level  factors  influence  students’  CIL?  

Multilevel modelling of school level variables (including   principals’   e-learning

leadership   practice   and   teachers’   ICT-using pedagogy factors) found more

frequent use of ICT by students in traditional learning tasks, and the extent to

which curriculum and assessment related obstacles hinder  the  school’s realization

of its e-learning goals as reported by principals as the only two statistically

significant  positive  predictors  of   students’  CIL achievement. These indicate that

students’   CIL   achievement   will   benefit   from  more   e-learning opportunities (as

opposed to e-teaching), as well as heightened leadership awareness of the need to

change curriculum and assessment practices in   order   to   realize   the   school’s   e-

learning goals.

There  were  only  two  other  significant  school  level  predictors  of  students’  CIL,  both of which were negative.

These were teachers’  reported  use  of  pedagogical  ICT  tools  by themselves and

the extent to which insufficient ICT hardware and software hindered  the  school’s  e-learning development, as reported by the principals.

These findings show that the most important influences on Hong Kong

students’  CIL  outcomes  are  those  at  the school level. Among these influences, the

single most important factor is the opportunities to use ICT in learning that

teachers provide to students. Furthermore, the findings show that having access to

computers and the Internet, as well as using them for personal and social

communication  purposes  per  se  do  not  affect  students’  CIL outcomes.

e-Learning and assessment designs to foster students’ CIL

Studies of ICT-enabled innovative pedagogical practices show that students are

much more likely to develop CIL if they engage in solving real-world problems in

collaboration with their peers. The key pedagogical design principles of e-learning

activities that  fostering  students’  CIL include:

x Give priority for ICT use to support students’  learning,  not  teaching; x Learning activities are learner-centric and inquiry- oriented;

x Learning tasks are extended in time, comprising multiple stages and/or

parts, with interim products generated in the process;

x Learning activities are authentic and related to students' daily life

experiences;

x Learning tasks are open-ended, providing opportunities for students to

make judgments;

x Provide learners with the opportunity to use ICT to access different sources

of information, organize, compare and contrast, analyse and integrate

information.

Assessment  practices  also  need  to  change  to  more  effectively  foster  students’  CIL.  

xxvi

Assessment as learning conceptualizes assessment as an integral part of learning and teaching, with students actively involved in this process. Assessment as learning occurs when students reflect on and monitor their own progress to inform their formulation of future learning goals and take responsibility for their own past and future learning. Traditional paper and pencil tests are not suitable for assessment as learning. Assessment rubrics and self-evaluation checklists are two of the most commonly used instruments in performance assessment. Rubrics are essential instruments for implementing Assessment as Learning. These are descriptive scoring tools for rating authentic student work qualitatively. Apart from rubrics, self-evaluation checklists are often used in the context of self- and peer- evaluation. A checklist provides a list of measurable categories and indicators for project, product and performance, allowing students to judge their own or peer’s  performance  and  determine  whether  they  have  met  the  established  criteria  of a task. The size of a learning unit often limits the complexity of the learning tasks that can be presented to students.

Longer learning units that span days or weeks involving tasks that need to be conducted both in school and at home often provide more opportunities for students to develop higher level CIL outcomes. These often require students to exercise skills in multiple CIL aspects in a meaningful and holistic fashion.

It is desirable for students to be provided with the same technology platform for learning in different subject matter contexts and tasks. It takes time for both teachers and students to get accustomed to the interfaces and functionalities of a new technology, creating additional cognitive and organizational burdens for all involved. This additional effort would be minimized if the same technology were used throughout the course of a  student’s  learning.

Conclusion

Hong   Kong’s   participation   in   the   ICILS   2013   Study   provided   us   with   a   good  overview  of  how  Hong  Kong  students’  proficiency  in  computer  and  information  literacy and how it compared with their international counterparts. The picture we get  from  the  many  different  analyses  is  consistent.  Our  students’  CIL  proficiency  compared to all economically developed countries participating in the study is low. This is despite the overall ICT development index as well as the level of ICT provisions in   schools   in  Hong  Kong   is   internationally   relatively  high.   Students’  access to computers and the Internet at home is almost 100%, which also compares extremely well with other participating countries.

Hong Kong has the second highest standard deviation in CIL score (second only to the City of Buenos Aires, Argentina), indicating a very wide spread in students’   CIL   proficiency.   On   the   other   hand,   students’   SES   background   has  amongst the lowest relationship with their CIL scores amongst all the participating countries.  This  indicates  that  school  factors  have  major  influence  on  students’  CIL  achievement.

xxvii

In Hong Kong, teachers are confident about their own basic ICT competence

but not in pedagogical uses of ICT. The levels of ICT use for student learning is

particularly low, even though this was found to the single most important positive

contributing  factor  to  students’  CIL  achievement.  In  Hong  Kong,  teachers  gave  low  emphasis   to   developing   students’   CIL   skills   and   principals   are   also   not   giving  much priority   to   the   use   of   ICT   to   support   students’   development   of   lifelong  learning   skills.   Hong   Kong   teachers’   participation   in   ICT-related professional

development was also relatively low.

Summarizing the ICILS 2013 findings, it is clear that there is a very strong

need for improvement in teaching, learning and assessment in schools to help our

students develop higher levels of CIL proficiency. This requires concerted

leadership efforts at the policy and school levels. This should be a prime priority

for the 4th IT in Education Strategy in Hong Kong, which was launched in August

2015. Findings from ICILS 2103 also triangulates with local ICT-related research

and development projects that only when students have the opportunity to use ICT

for learning (i.e. e-learning as opposed to e-teaching) will they really be able to

advance in CIL proficiency. Also, the more challenges CIL aspects such as

evaluating, managing and sharing information can only be effectively fostered

through e-learning in open-ended, inquiry based learning tasks involving

authentic, real life problems and are extended over days or weeks. Assessment

practices need to be changed such that the focus is on helping students understand

the criteria of assessment as well as the benchmarks for different levels of

achievement for each criterion. The availability of an integrated, online learning

and assessment support platform that can incorporate peer- and self-assessment

using  rubrics  and  checklists  would  go  a   long  way  towards  supporting  students’  development of CIL through their learning throughout the school curriculum.

1

Chapter 1

Computer & Information Literacy and its Assessment

Serious discussions at the policy level about computer and information literacy (CIL) as an important student learning outcome began to appear in the West in the 1990s. In Hong Kong, around the same time, two policy documents regarding this were also published by Education and Manpower Bureau as part of the Second IT in Education Strategy of the HKSAR government. These policy documents have led  to  a  series  of  curriculum  initiatives  to  foster  students’  CIL  and  have  also  raised  issues concerning how CIL can actually be assessed. The International Computer and Information Literacy Study (ICILS), which was conducted under the auspices of the International Association for the Evaluation of Educational Achievement (IEA) in 2013, was the first international comparative study that assesses  students’  performance in CIL. Hong Kong was among the 21 countries/systems, which

participated in ICILS 2013. The aim of this book is to report on the findings from ICILS 2013, which are of particular relevance to Hong Kong in comparison with its international  peers.  Students’  learning outcomes can be influenced by personal and family background as well as school and system level factors, which are also explored within the design of the Study. In order that the research findings can be easily understood by

teachers, principals, teacher educators, policy-makers, parents, e-Learning technology providers and anyone who is concerned about student learning in the 21st century, this publication is not structured as a formal research report. Instead, that the focus is to bring in the relevant local and international policy and contextual backgrounds to highlight the key concepts and findings relevant to the education community and the general public. For readers who are interested to learn the specific research methodology and data analyse techniques adopted in the study, they can refer to the ICILS 2013 International Report (Fraillon et al., 2014) and the associated Technical Report (Fraillon et al., 2015).

Figure 1.1 Logo for European Computer Driving Licence Foundation (ECDL)

2

This book is structured around five chapters.

Chapter 1 starts with a brief review of the

historical development of the concepts of CIL

both in an international context and in Hong

Kong. This is followed by a section introducing

respectively the assessment framework adopted

in ICILS 2013, and the assessment design of the

performance test booklets used in the study to

provide coverage for the various CIL aspects of

the  framework.  Chapter  2  reports  on  students’  performance in ICILS 2013, with a special focus

on  Hong  Kong  students’  performance,  giving  a  detailed breakdown of their strengthens and

weaknesses in comparison with two selected

high-performing, developed countries,

Australia and Korea. Chapter 3 focuses on

how   students’   personal   and   family   background   as   well   as   ICT   use   experience  within and outside schools influenced their CIL learning outcomes. The findings

reported in this chapter draws on analyses of the Hong Kong, International,

Australian and Korean data collected through the survey questionnaire

administered to students immediately after they submitted their CIL performance

test. The sampling method and participation rates for students are also explained.

Chapter 4 reports on analyses of the data collected from the teacher and

principal  questionnaires  to  examine  how  school   level   factors   influence  students’  CIL outcomes. A brief introduction of the principles underpinning the design of

the questionnaires and the sampling methods for the principal and teacher

questionnaires is also reported.

As  a  main  objective  of  this  Study  is  to  help  contribute  to  improving  students’  CIL, the core focus of Chapter 5 is to recommend and describe learning and

assessment  designs   that  will   foster   students’  CIL.  The  designs  described in this

chapter are selected from case studies conducted by CITE in past and on-going

research projects on e-Learning in Hong Kong schools.

1.1 Computer and Information Literacy: a Brief History

Computer and Information Literacy actually comprises two interrelated types of

literacy: Computer Literacy (CL) and Information Literacy (IL). CL is the original

term used in this field. The European Computer Driving Licence (ECDL) launched

in 1995 is among the earliest certification programs of its kind, which focuses on

individuals’  technical  skills  in  using  basic  applications,  such  as  Office  and  email,  as well as their basic knowledge about computers and information technology (IT)

security.

Figure 1.2 National Education Technology Standards (NETS), 1998

3

Underpinning this certification programme is the idea that the possession of basic computer skills is just as necessary in a digital society as being literate in reading and writing.

In  addition  to  the  policy  perspective,  concerns  about  individuals’  computer  literacy have also been raised from the education and businesses sectors. They suggest  that  students’  computer-related competence should go beyond the use of basic digital productivity and communication tools (CL), and include the use of IT as a tool for research, problem-solving and decision-making. Further, awareness of the social, ethical and human issues related to the use of digital technology are also deemed important for ordinary citizens. In 1998, the National Education Technology Standards (NETS) published by the US-based International Society for Technology in Education (ISTE) adopted this approach, even though the specific word  “literacy”  was  not  used.  As  the  required  basic  skills  are  constantly  evolving  as a result of social and technological development, it is not surprising to see that the   NETS   was   updated   in   ISTE’s   2007   report.   Table   1.1   shows   a   comparison  between the key elements addressed in these two standards.

Table 1.1 The key standards areas specified in the 1998 and 2007 NETS

1998 2007

Technology Productivity Tools Creativity and Innovation

Technology Communications Tools Communication and Collaboration

Technology Research Tools Research and Information Fluency

Technology Problem-solving and Decision-Making Tools

Critical Thinking, Problem Solving, and Decision Making

Social, Ethical, and Human Issues Digital Citizenship

Basic Operations and Concepts Technology Operations and Concepts

Despite a few similarities between the two standards, the latter clearly has a

stronger emphasis on the execution of higher order abilities in the use of IT, such as creativity, innovation, critical thinking and collaboration, as well as on the importance of digital citizenship.

Professional library associations have been playing an active role in promoting the concept of Information Literacy (IL). The focus of IL is not on computing knowledge or technical skills, but rather on the ability to source, organize, evaluate and make appropriate use of information in different contextual situations for learning and problem solving. For example, the information literacy framework proposed by SCONUL (the Society of College, National and University Libraries, UK) titled Information skills in higher education: a SCONUL position paper (1999), comprise  “seven  pillars”  of  competence  (see  Figure  1.3).  Performance  in  each  pillar  of  competence  ranges  from  “Novice”  to  “Expert”.  SCONUL’s  2011  revised version was largely similar to the previous one.

4

The only difference is in the naming of the seven pillars, which was shortened

to one word per pillar for easier

memorisation: identify scope, plan,

gather, evaluate, manage and present.

The global interest in information

literacy education was spurred by the

growing recognition that students need

to be prepared for lifelong learning to

participate effectively in a society in

which information and knowledge are

increasing at an exponential rate.

In the IL Framework published by

the Australian and New Zealand

Institute for Information Literacy, IL is

conceptualized as personal

empowerment (Bundy 2004) and

depicted as a subset of independent

learning skills nested within lifelong

learning skills (see Figure 1.4).

1.2 Computer and Information Literacy in the Hong Kong School Curriculum

In Hong Kong, Computer Studies was introduced into the upper secondary school

curriculum as an elective subject in 1982. Following the launch of the first

Information Technology in Education Strategy (EMB, 1998). The Information

Technology Learning Targets (ITLT) were put forward in 2000 by a working group

formed under the Curriculum Development Institute. The ITLT documents the

knowledge, skills and attitudes that students are expected to obtain at five different

key stages in using IT tools respectively.

(in SCONUL (2004), p. 4) Figure 1.3 The seven pillars of information literacy according to SCONUL.

Figure 1.4 Relationship of IL to lifelong learning (Bundy, 2004, p. 5)

5

Figure 1.5 lists the eight modules included in the ITLT (EMB, 2000) document for key stages I and II. In the education reform launched in 2000, IT skills, defined

as skills “to  seek,  absorb,  analyse,  manage  

and present information critically and intelligently in an information age and a digitised  world”  was included as one of the nine generic skills (see Figure 1.6). It is thus clear that there is a conscious shift in focus from ICT technical skills to information skills, within the educational policy arena as well as in the wider educational community. Figure 1.7 shows the covers of the key education reform consultation documents published in 2000.

In 2004, the Education and Manpower Bureau commissioned a consortium of scholars from four teacher education institutions (BU, CUHK, HKIEd, UHK) to develop an information literacy framework for Hong Kong students as part of the objectives for the Second Information Technology in Education Strategy (EMB, 2004). Figure 1.8 shows the cover of the report published in 2005 and the diagrammatic representation of the concept of IL put forward in this document.

In 2007, as part of the EDB evaluation of the effectiveness of the Second Information Technology in Education Strategy (2004-2007), the Centre for Information Technology in Education (CITE) at the University of Hong Kong was commissioned by EDB to conduct an Information literacy Performance Assessment Study (ILPA for short) of Primary 5 and Secondary 2 students on a random sample of government- funded schools.

The details of the assessment will be introduced in the next section, it is important to note here that the description of information literacy in the EDB (2005) document was at a level of conceptual that was too abstract to serve the purpose of an assessment framework that can be operationalized into the design of performance assessment tasks. Based on a thorough literature review at the time, the CITE project team adopted the seven dimensions that ETS (2003) uses to design its information literacy assessments in higher education together with ethical use of information to form the eight dimensional framework for designing the assessment tasks in the evaluation.

(EC, 2000, p.24) Figure 1.6 The nine generic skills specified in the Reform Proposal

(EMB, 2000, p.20) Figure 1.5 Modules included in the Information Technology Learning Targets guideline

Using E-mail Word Processing Using the Internet Writing with a Computer Drawing with a Computer Joy to the Computer World Calculating and Charting with Spreadsheet Learning to control a Computer through Logo

6

Figure 1.7 Consultation documents published by the Education Commission in 2000 with a focus on preparing students for lifelong and lifewide learning

Figure 1.8 The Information Literacy Framework released by the EMB in 2005 and the graphical representation for information literacy used in the document

The results of the assessment (Law, Lee and Yuen, 2010) showed generally

unsatisfactory performance of the assessed students; and large disparities in students’  achievement  across schools (Law et al. 2007).

In view of these findings, the EDB further commissioned CITE to conduct a one year design-based research and development project with primary and secondary school teachers, in order to develop (1) curriculum designs that promote students’   IL   and   (2)   tools   for   assessing   students’   IL   outcomes  (http://iltools.cite.hku.hk/ referred to as ILTOOLS for short).

7

The   students’   IL   framework   (EMB,  2005)   recommends   that  “IL  assessment  should   be   formative   and   developmental   […]   [and   should   be]   designed   for  developing  the  capability  of  learners  in  learning  different  subject  disciplines  …”  (p. 18). As such, the ILTOOLS project was commissioned to focus on IL within the Science KLA.

In order to support easy operationalization of the EMB (2005) IL framework for curriculum and assessment integration, the CITE project team crystalized students’   IL   into   an   eight-element framework used in ILTOOLS (define, access, manage, integrate, create, communicate, evaluate and ethical use) for working with information, and provided explicit links between this framework and the 12 inquiry skills suggested in the Science curriculum guide (EDB 2006). Figure 1.9 shows a graphical representation of the IL framework and the scientific inquiry skills framework produced by the ILTOOLS team.

(http://iltools.cite.hku.hk/). Figure 1.9 Graphical representations of the Information Literacy and Scientific Inquiry frameworks used by the ILTOOLS team

1.3 Assessing Computer and Information Literacy

Assessing computer and information literacy often involve test takers to work in simulated environments. As the Internet is becoming the major source of information,  assessing   test   takers’  operation  proficiency  on  a   simulated   Internet  environment is one of the major means of CIL assessment.

8

In the case of the European Computer Driving Licence, which focuses on

technical skills in basic computer applications, testing is arranged on a module

basis. Testing is conducted on a simulated Windows/Microsoft Office

environment, which records mouse movements and keystrokes and reports the

results of the test immediately upon test completion.

1.3.1 European Computer Driving Licence

Since 2005, the Australian National Assessment Program has been conducting ICT

literacy assessment on a sample of Year 6 and Year 10 students every three years

(referred to as the NAP-ICT literacy

assessment). In this assessment, ICT

literacy is considered as a generic ability

of the students (i.e. independent of the

subject domain). Three strands are

included in this assessment framework,

namely, working with information,

creating and sharing information, and

using ICT responsibly. These strands are

assessed through six processes as

depicted in Figure 1.10. The NAP-ICT

literacy assessment was delivered in

schools via online modules supported by

purpose-built software applications. The

assessment modules usually comprise a

sequence of simulated tasks in a variety

of response formats, such as multiple choice, drag and drop, execution of simple

software commands, short constructed text responses and construction of

information products such as a poster. The NAP-ICT literacy assessment

instruments were designed and developed by the Australian Council for

Educational Research (ACER), which is also the team that took responsibility for

designing the ICILS assessment instrument. Hence, there is great similarity

between the NAP-ICT literacy assessment and the ICILS assessment. The latter will

be described in the next section.

As aforementioned, information literacy, as a generic ability, could enhance

students’   learning   outcomes   in   different   subject   areas   according   to   the   IL  framework in Hong Kong. The assessment of IL, therefore, can be subject

dependent. However, there are no well-known examples of performance

assessment of IL in subject-based contexts. The Program for International Student

Assessment (PISA) conducted by OECD introduced computer-based assessment

(CBA) as an additional option for participating countries since 2009. In PISA 2012,

countries could opt to participate in CBA of problem-solving, mathematics and

reading literacy.

Figure 1.10 The Australian ICT Literacy Assessment framework (ACARA, 2015, p.4)

9

However, the PISA CBA does not assess subject-specific information literacy.

It   only   assessed   students’   ability   to   accomplish   subject-based tasks, using

computers as the medium instead of paper. Exactly the same set of competence is

being assessed in the paper-based and computer-based assessments of Reading

and Mathematics. For problem solving, the assessment framework of CBA is

essentially the same as that of paper-based assessment, except that for CBA the

nature of the problem situation can be interactive (i.e. not all information is

disclosed and the student has to uncover some of the information by exploring,

such as clicking particular buttons).

1.3.2 Computer-Based Assessments in PISA

As   mentioned   in   the   previous   section,   the   assessment   of   students’   IL   was  conducted by the CITE team IN 2007 as part of the evaluation of the effectiveness

of the Second IT in Education Strategy in Hong Kong (Law et al., 2007). The goal

and design of the study, Information Literacy Performance Assessment (ILPA),

followed  ETS’s  (2003)  IL  framework  and  assessed  IL  in  both  generic  and  subject-specific problem solving contexts. The evaluation was carried out at two grade

levels. At the primary 5 level, students were assessed on their generic IL skills and

their ability to use IL tools to solve subject-specific problems in Chinese Language

and Mathematics that they would otherwise not be able to do based on the P. 5

curriculum for these two subjects. At the secondary 2 level, the evaluation included

assessing  students’  generic  IL  skills  and  their  ability  to  use  IL  tools  to  solve  subject- specific problems in Chinese Language and Science that are beyond the respective

secondary 2 curriculum requirements. Hence, three tests were administered at each

grade level. The same generic IL test was administered to Primary 5 and Secondary

2 students in order to compare the generic skills levels for these two levels of

students. As a result, a total of five different tests were developed in the ILPA

Study.

Figures 1.11 to 1.13 present examples of the questions in ILPA, which were

used  to  assess  students’  ability  to  integrate information in the generic mathematics

and science tests respectively. Integrate information relates to the ability to interpret

and represent information by using ICT tools to synthesize, summarize, compare

and contrast information from multiple sources. As shown by the examples,

although   the   three   questions   were   all   designed   to   assess   students’   ability   to  integrate information, they differ in the kinds of tools adopted and the

understanding needed to complete the question.

The question shown in Figure 1.11 asks students to create a PowerPoint

presentation on a one-day trip in Hong Kong that includes two scenic spots

suitable. Students can use the Internet to search for information about the

appropriate places to visit, their opening hours, traffic routes, and descriptions

about the scenic spots in those places.

10

Figure 1.11 An item in the Primary 5 and Secondary 2 ILPA generic test that assesses the ‘integrate’  dimension

In   the   ILPA   Mathematics   test,   the   question   assessing   students’   ability   to  

integrate information for mathematical problem solving was related to geometry (see Figure 1.12). Students were asked to work out the maximum area that can be encompassed by a rectangle with a fixed perimeter.

Students were given a Java applet to explore the areas inside differently configured rectangles with the same perimeter. With the applet, Primary 5 students should be able to arrive at the optimal configuration (a square) if they can record the areas when the length of the rectangle is systematically varied. This problem can only be solved analytically by using calculus, which would prove challenging even for senior secondary students.

In this item, students need to create a PowerPoint presen tation on a PLAN for a day trip for the elderly to visit in Hong Kong. They were assessed on their ability to synthesize, compare and contrast information as well as summarizing information from muliple digital sources.

11

Figure 1.12 An  item  in  the  Primary  5  ILPA  mathematics  test  that  assesses  the  ‘integrate’  

dimension In the ILPA Science test, the task assess Secondary   2   students’   ability   to  

integrate information in scientific problem solving was about population dynamics in pond ecosystems.

In the question, shown in Figure 1.13, students were asked to use a visual dynamic simulation tool to explore and observe how adding a foreign species to a local pond will affect the population size of different species living in the pond ecosystem over a given period of time. With the simulation tool, students can observe changes in the number of species through iconic visualization of the species in the pond as well as through the line graphs showing the actual numbers of each species over time. Originally, there were four species living in the simulated pond ecosystem: water plants, shrimps, fish and ducks.

In this item, students can manipulate an interactive applet to observe changes in the area of a rectangle with a fixed length perimeter using the applet.  Students’  performance is assessed on the comprehensiveness of the  students’  manipulations and observations, and the correctness of the students’  interpretations.

12

In this task, students were provided with a dynamic simulation tool to explore the effect of adding red shrimps (a foreign species) to the local pond ecosystem. Students were assessed on their ability to make observations of the critical changes in population through both the iconic grid and the graph (see figure below) and their ability to make appropriate interpretation and conclusion.

This is the screen of the simulation pond ecosystem for students to conduct “ecological  experiments”.  The upper part of the main screen shows the changes in the population size of the different species in the pond in iconic form, and the lower section shows the same informa- tion in graphic format. Instruction is given in the lower left-hand corner.

Figure 1.13 Screen dumps for a task in the Secondary 2 ILPA Science test that assesses the  ‘integrate’  dimension.

13

At the beginning of the task, students were asked to observe and describe how the numbers of different species change over 200 days. Then “a  visitor  brought  20 red shrimps to add to the pond. He thinks this will increase the biodiversity of this pond ecology.” At this point, 20 red shrimps also appear in the simulated ponds.

The students were then asked to observe the changes over another 600 days. They could also click on the species icons on the upper left hand panel to learn about their behavior. They were then asked to answer four questions, three of which pertain to integrating information, as follows:

Why did most (or all) of the shrimps die? Why did most (or all) of the fish die? What are the possible impacts of adding a foreign species to an ecosystem?

The topic of this task is the concept of competition among species that share the same niche. Hence, the introduction of foreign species to an indigenous ecology may lead to the extinction of not only the species they compete with, but also those that feed on the local species that are driven to extinction. This is a very challenging task even for high schools and university students. Using the simulation tool, students would be able to collate their observation of the relations in trend data across the species and to point out that the indigenous shrimps slowly die off as a result of the increase of the red shrimps. They found that the fish, which feed on the indigenous shrimps (but not the red shrimps), actually went extinct even before the indigenous shrimps. Hence, just as in the Mathematics task above, in this example, by using the simulation tool, Secondary 2 students with the requisite IL skills were able to learn about some important concepts in science, which would otherwise be inaccessible.

The above three example assessment tasks show that different kinds of items need  to  be  designed  to  assess  students’  ability  to  integrate  information  as  a  generic  IL skill or as IL skills in specific subject learning contexts. Similarly, each of the ILPA generic and subject specific IL tests (five different tests were designed and administered in the Study as described earlier) contain items that assessed each of the seven IL dimensions, which include define, access, manage, create, communicate and evaluate, in addition to integrate, which is already described. Findings  from  ILPA  show  that  when  students’  IL  performance  in  the  different  tests  is   correlated,   students’   prior   learning   experience   with   subject-specific tools for learning purposes matters (Law et al., 2007).

1.4 Definition of CIL in ICILS

ICILS 2013 only assesses CIL as a generic skill, and does not make any explicit assumption about whether performance in IL is subject matter dependent. The assessment framework used in ICILS measures CIL according to two broad conceptual categories named strands: collecting and managing information and

14

producing and exchanging information (Fraillon, et. al., 2014). Each strand is further categorized into skills categories labelled as aspects. Table 1.2 provides a brief description of the CIL strands and aspects. Table 1.2 The Computer and Information Literacy Framework adopted in ICILS

CIL strand & aspect Assessment focus Strand 1: collecting and managing information

1.1 Knowing about and under- standing computer use

Basic technical knowledge and skills needed to work with information using computers.

1.2 Accessing and evaluating information

Ability to find, retrieve, and make judgments about the relevance, integrity, and usefulness of computer-based information.

1.3 Managing information Ability to adopt and adapt schemes of information classification and organization to arrange and store information for efficient use and/or re-use.

Strand 2: Producing and exchanging information

2.1 Transforming information

Ability to use computers to vary the presentation of information to achieve clarity suitable for specific audiences and purposes.

2.2 Creating information Ability to use computers to design and generate new or derived information products for specific audiences and purposes.

2.3 Sharing information Ability to use computers to communicate and exchange information with others.

2.4 Using information safely and securely

Understanding of the legal and ethical issues of digital communication from the perspectives of both information producer and consumer.

If we compare the above CIL assessment framework with the assessment

framework adopted in ILPA, which was based on the IL framework for students in Hong Kong published by the EDB (EMB, 2005), we can see that they are in fact very similar. Table 1.3 provides a comparison across the two assessment frameworks.

15

Table 1.3 A comparison of the IL assessment framework adopted by the ILPA Study in

Hong Kong (Law et al. 2007) and that used in ICILS

ILPA Assessment Dimensions ICILS assessment strands & aspects

Defining information needs 1.2 Accessing and evaluating information

Accessing information 1.2 Accessing and evaluating information

Managing information 1.1 Knowing about and understanding computer use

1.3 Managing information

Integrating information 2.1 Transforming information

Creating information 2.2 Creating information

Communicating information 2.3 Sharing information

Evaluating information 1.2 Accessing and evaluating information

Ethical use of information* 2.4 Using information safely and securely

1.5 Test Administration and Module Design in ICILS

Data collection for ICILS in schools took place between February and June 2013.

Twenty students were randomly selected from each participating school to take

part in the CIL performance assessment. Assessment took place in the school

computer rooms, and the test instruments were delivered using software purpose-

designed by the International Study Centre and administered on USB drives

connected to the school computers. The test administration was delivered through

USB drives instead of via the Internet to ensure a uniform assessment environment

regardless of the quality of the Internet connection of the school computers used.

Further, by setting all Internet related operations to take place in a simulated web

environment, the assessment can avoid the potential problem of differential

exposure to resources and information outside of the test environment.

Four 30-minute test modules were constructed for the purpose of CIL

assessment in ICILS. A module comprised a set of questions and tasks based on a

theme familiar to students, following a linear narrative structure. Each module was

designed to have the same structure: starting with a number of short independent

tasks (usually requiring less than one minute), followed by a large task that was

expected to take 15-20 minutes to complete. The themes, and hence the titles of the

four modules are: After-School Exercise, Band Competition, Breathing, and School Trip.

Each participating student was randomly assigned two of the four modules to

complete.

A sample screen layout for the test administration is shown in Figure 1.14.

The main space is occupied by the sample computer screen that gives the context

for the computer-based task.

16

The bottom part of the screen gives instructions on what the student is required to do. The narrow upper right panel gives a visual indication of how many tasks the student has completed (red boxes) relative to the entire set of tasks (green boxes).

Figure 1.14 A sample screen in an ICILS test module showing the functions for different sections of the screen

There are two item formats in the short tasks found in the ICILS test modules:

close-ended and open-ended. Responses to close-ended short tasks include multiple choice and test environment operations (i.e. mouse clicks to carry out certain tasks in the simulated test environment), which were recorded and scored automatically. Open-ended short tasks required students to enter their responses in their own words, and were scored by trained local expert scorers according to the scoring rubric. Slightly more than half of the score points were allocated to the satisfactory completion of the large task for each of the modules. The work submitted for the large tasks were scored according to the scoring criteria, which were presented to the students at the start of the large task, and remain accessible throughout the duration of the task. After-School Exercise, one of the four assessment modules of ICILS, is made public and a computer-based demonstration can be accessed at http://www.iea.nl/icils_2013_example_module.html.

17

In order to illustrate the structure of the test modules, a summary of the short tasks and large task involved in the After-School Exercise module are described here. Table 1.4 gives a brief description of the short tasks, responses required and the assessed aspect according to the CIL framework.

Table 1.4 Overview of the short tasks in the After-School Exercise module Task no. Description Response type Assessed

aspect

1 Given a sample email, identify the recipients. MC*, tick all that apply 2.3

2 Follow the URL given in plain text in the email message to access a site.

Copy & paste URL to task bar of simulated web 1.1

3

In creating a user account on a public website, identify which personal information is most risky to include in a public profile.

MC*, select one. 2.4

4 Given an email alert on the simulated web, open the email via the hyperlink.

Click the hyperlink on the screen. 1.1

5

You received an email that tries to trick you into giving your password to the sender (phishing email). Identify the suspicious feature in the sender address.

Type explanation that identifies the suspicious domain name.

2.4

6 Based on the email message in Task 5, identify the suspicious feature in the greeting.

Explanation: there is no personal identifier. 2.4

7 Based on the email message in Task 5, identify the suspicious feature in the password reset URL link.

Explanation: the URL is not the correct one. 2.4

8 Explain one problem that may result from  making  one’s  email  address  publicly available.

Explanation: unwanted email, lose privacy 2.4

9

Given a document on a social networking site, make use of the sharing settings to give  “can  edit”  access  to  a  specified  person

Use the sharing function to add new user and set access  right  to  “can  edit”.

1.1

* MC refers to multiple choice objective type questions. In this test module, students were asked to design a poster to advertise the

After-School Exercise program and attract fellow students to participate. The exercise program should take about 30 minutes and should be suitable for students over the age of 12. The students had to select a suitable exercise by visiting the simulated websites provided.

18

They should then design their poster in the simulated poster design environment for which they had supposedly applied for an account in one of the short tasks earlier.

Figure 1.15 show screen captures of the poster design environment, the simulated website containing exercise information and graphics, as well as the instructions and assessment criteria for the poster to be constructed.

a. Poster design environment.

b. Website with graphics and text

about three exercises.

c. Instructions and assessment criteria about the large task.

Figure 1.15 The poster design environment, information resources, instructions and assessment criteria provided to students for working on the large task

Each poster was scored according to nine criteria: appropriateness of the

poster title; layout of images; formatting design of text; colour contrast of text consistency of colour scheme used for text, graphics and background; editing of information text from website; completeness of information; persuasiveness of poster; and full use of the entire poster space.

Five of the criteria had a maximum score of two, reflecting two levels of performance. Table 1.5 lists the performance expectation for each of the score points and the related CIL aspect that is being assessed.

19

Table 1.5 Summary of the performance expectation and associated CIL aspect for each of the score point criteria in the After-School Exercise long task module

Criterion Score/ max. score

Performance expectation Assessed aspect

1 Title design 1/2 A relevant title has been added and placed in a

prominent position. 2.2

2/2 A relevant title has been added and formatted to make its role clear. 2.1

2 Image layout 1/1 One or more images are well aligned with the other elements on the page and appropriately sized. 2.2

3 Text layout & formatting

1/2 Formatting tools have been used to some degree to show the role of the different text elements. 2.2

2/2 Formatting tools have been used consistently throughout the poster to show the role of the different text elements.

2.2

4 Colour contrast 1/2 The text mostly contrasts sufficiently with the

background to support reading. 2.2

2/2 There is sufficient contrast to enable all text to be seen and read easily. 2.1

5 Color consistency 1/1 The poster shows evidence of planning regarding the use of color to denote the role of the text, background, and images in the poster.

2.3

6 Information adaptation

1/2 Some useful information has been copied from the resources and edited to improve ease of comprehension and relevance.

2.3

2/2 The relevant key points from the resources have been rephrased using student's own words. 2.3

7 Information completeness

1/2 Two of the three required pieces of information about the program (when, where, and what equip- ment is required) have been included in the poster.

1.2

2/2 All required information about the program (when, where, and what equipment is required) has been included in the poster.

1.2

8 Persuasiveness 1/1 Uses some emotive or pesuasive language to make the program appealing to readers. 2.1

9 Use of full page 1/1 Uses full page when creating poster 2.1

Except for the two criteria connected with colour use, which were machine

scored, scoring of the posters created by students was carried out by locally trained expert scorers according to the scoring rubrics, similar to the case with the open-ended short tasks.

20

1.6 Summary

This chapter has provided a historical overview of the development of the key concepts of CIL both in an international context and in Hong Kong. They are computer literacy, information literacy, and computer and information literacy. The CIL framework used in the assessment design in ICILS is similar to the one used in the Hong Kong ILPA Study, which was developed based on the IL framework published by the US Educational Testing Service (ETS. 2003).

While there are several survey instruments designed to gather information that  may  reflect  an  individual’s  CIL,  this  chapter  focused  only  on  those  that  involve  computer-based performance assessment of CIL. We expressly differentiated general computer-based performance assessments (e.g. CBA in the PISA studies) from those with particular focus on CIL (such as ICILS or ILPA). Further, based on the IL frameworks published by well-known  library  associations  and  the  students’  IL framework published by the EDB in 2005, we pointed out that IL should not be viewed only as a generic skill, but also as a skill that supports learning in different disciplinary areas. ICILS assesses CIL only as a generic skill, although it is conceivable that assessment of IL can be conducted both as a generic skill in general problem solving and in subject-specific learning contexts.

The ICILS test design gave a stronger emphasis on Strand 2 aspects, which were assessed through a large task set within a real-life context familiar to students. Machine scoring was limited to close-ended short tasks that require multiple-choice answers or test environment operations, and scoring of constructed artefacts with reference to colour contrasts. The majority of the tasks were scored by trained human experts based on set criteria. The quality of scoring was ensured through inter-coder reliability check on doubled scored samples.

21

Chapter 2

Hong Kong Students’ Performance in ICILS

A total of 21 countries and educational systems participated in ICILS 2013. These include, besides Hong Kong, Australia, the City of Buenos Aires (Argentina), Chile, Croatia, the Czech Republic, Denmark, Germany, Korea, Lithuania, the Netherlands, Norway, Newfoundland and Labrador (Canada), Ontario (Canada), Poland, the Russian Federation, the Slovak Republic, Slovenia, Switzerland, Thailand, and Turkey. The student population for ICILS was defined as students in Grade 8, provided that the average age of the students was at least 13.5 at the time when the assessment was carried out. If the average age of Grade 8 students was below 13.5, Grade 9 students would then become the targeted population. Norway was the only country that had Grade 9 students tested in ICILS 2013. In Hong Kong, Secondary Two was the targeted student population for the Study.

In order to provide the necessary background for understanding and interpretation of Hong Kong students’  performance in ICILS, this chapter begins with a brief introduction of the sampling method and criteria of ICILS. This is followed by an illustration of how the CIL scores were computed and linked to different levels of CIL proficiency. Next, the CIL scores and proficiency profiles of all participating systems will be reported, together with an in-depth analysis of the strengths and weaknesses in  Hong  Kong  students’  CIL  performance. For reference and comparison purposes, we also analyse and report on the results of Australia and Korea, which have a similar level of economic and social development as Hong Kong.

2.1 Sampling Method and Participation Rates

As mentioned in Chapter 1, the targeted student population in ICILS is Grade 8, which is equivalent to Secondary 2 in Hong Kong. Student sampling was conducted in two stages. During the first stage of sampling, 150 schools1 were randomly sampled using PPS procedures (probability proportional to size as measured by the number of students enrolled in a school).

1 The number of schools required in the sample to achieve the necessary statistical precision were estimated on the basis of national characteristics, and ranged from 138 to 318 across countries. Countries were asked to plan for a minimum sample of 150 schools, which was adopted as the sample size in Hong Kong.

22

Twenty students were then randomly sampled from all students enrolled in the targeted grade in each sampled school. In schools with fewer than 20 students in Grade 8, all students were invited to participate. During the sampling process, replacement schools with sizes closest to the sampled school were also sampled to serve as replacements when the sampled school refused to participate.

The participation rates required for each country were 85% of the selected schools and 85% of the selected students within the participating schools, or a weighted overall participation of 75%.

In Hong Kong, 118 schools participated in the student performance assessment and survey, giving a weighted school participation rate of 77%. Within the participating schools, a total of 2089 students participated in the online performance assessment, giving a weighted student participation rate of 89.1%. The weighted overall student participation rate was computed to be 68.6%, which was below the 75% required by IEA for the results to be considered as meeting the statistical   precision   required   for   international   comparison   of   students’  achievement.

Hence, the Hong Kong results are listed in the international report under the category  “countries  not  meeting  sample  requirements”, alongside with Denmark, Netherlands and Switzerland.

2.2 Computation of CIL Scores and Countries’  Performance

The four test modules comprise a total of 62 discrete questions and tasks, with a total of 81 score points. As introduced in Section 1.5, only a limited set of tasks was machine scored, and the remainder were scored by trained expert scorers according to set criteria. Just under half of the score points were derived from the short tasks. The test modules did not give the same emphasis to each of the seven aspects in the assessment framework (described in section 1.4). Instead, about twice as many score points were assigned to Strand 2 (producing and exchanging information) as compared to Strand 1 (collecting and managing information). The first three aspects in Strand 2 were primarily assessed through the large tasks.

The allocation of score points to the aspects and strands in the framework is presented in Table 2.1. The established practice in all IEA studies of student achievement is to use Rasch IRT (item response theory, Rasch, 1960) to construct the achievement scale with a mean of 500 and a standard deviation of 100.

The same process was applied to compute the cognitive scale of CIL scores with   a   mean   of   500   and   a   standard   deviation   of   100,   based   on   students’  performance data on the 62 questions and tasks in the four modules and equally weighted national samples.

As each student was randomly assigned one of the twelve possible combinations of two out of the four modules (the same two modules presented with a different order of appearance was considered as a different combination in order to control for the influence of module sequence on difficulty), plausible value

23

methodology was used to derive summary student achievement statistics, and to estimate the uncertainty of the measurement (von Davier, Gonzalez, & Mislevy, 2009; Fraillon et al. 2015).

Table 2.2 presents the CIL average scores for the 21 participating countries and systems. It also provides, for reference purposes, the respective ICT development index and student-computer ratios. Table 2.1 Allocation of module score points to the ICILS assessment framework

CIL strand & aspect % toward overall CIL score

Strand 1: Collecting and managing information

1.1 Knowing about and understanding computer use 13%

1.2 Accessing and evaluating information 15%

1.3 Managing information 5%

Strand 1 subtotal 33%

Strand 2: Producing and exchanging information

2.1 Transforming information 17%

2.2 Creating information 37%

2.3 Sharing information 1%

2.4 Using information safely and securely 12%

Strand 2 subtotal 67%

As shown in Table 2.2, the mean CIL score of Hong Kong students is 509,

which  is  slightly  above  the  international  mean  of  500.  Since  Hong  Kong  students’  overall participation rate is below 75%, comparing their performance with those in other participating countries is statistically inappropriate. In addition, it can also be seen that the CIL score of Hong Kong students is relatively low among all of the economically  developed  participating  educational  systems.  As  Hong  Kong’s  ICT  Development Index is ranked fifth among the 21 participating systems, this should not be a contributing factor to the relatively low CIL performance.

In Table 2.2, another factor worth noting is the standard error in CIL scores. The standard error in the mean CIL score of Hong Kong students is the largest among those in all participating systems. The large standard error reflects large variations in the CIL achievement among Hong Kong students. What were the factors  that  contributed  to  such  a  large  diversity  in  the  students’  performance? To what   extend  did   the   students’   family   background   and   their   school   and   teacher  level factors contribute to their CIL achievement? Answers to these questions are explored in the next two chapters.

24

Ta

ble 2

.2 C

ount

ry a

vera

ges f

or C

IL, y

ears

of s

choo

ling,

ave

rage

age

, IC

T In

dex,

stud

ent–

com

pute

r rat

ios a

nd p

erce

ntile

gra

ph

Educ

atio

nal s

yste

ms

ICT

Dev

elop

men

t In

dex

scor

e (a

nd

coun

try

rank

)Cz

ech

Repu

blic

814

.355

3(2

.1)

▲6.

40 (3

4)10

(0.3

)A

ustr

alia

814

.054

2(2

.3)

▲7.

90 (1

1)3

(0.3

)Po

land

814

.853

7(2

.4)

▲6.

31 (3

7)10

(0.5

)N

orw

ay (G

rade

9)

914

.853

7(2

.4)

▲8.

13 (6

)2

(0.1

)Re

publ

ic o

f Kor

ea8

14.2

536

(2.7

)▲

8.57

(1)

20(2

.3)

Ger

man

y8

14.5

523

(2.4

)▲

7.46

(19)

11(0

.8)

Slov

ak R

epub

lic8

14.3

517

(4.6

)▲

6.05

(43)

9(0

.5)

Russ

ian

Fede

rati

on8

15.2

516

(2.8

)▲

6.19

(40)

17(1

.0)

Croa

tia

814

.651

2(2

.9)

▲6.

31 (3

8)26

(0.8

)Sl

oven

ia8

13.8

511

(2.2

)▲

6.76

(28)

15(0

.5)

Lith

uani

a8

14.7

494

(3.6

)5.

88 (4

4)13

(0.7

)Ch

ile8

14.2

487

(3.1

)▼

5.46

(51)

22(4

.7)

Thai

land

813

.937

3(4

.7)

▼3.

54 (9

5)14

(0.9

)Tu

rkey

814

.136

1(5

.0)

▼4.

64 (6

9)80

(16.

0)

Educ

atio

nal s

yste

ms

not

mee

ting

sam

plin

g re

quir

emen

tsD

enm

ark

815

.154

2(3

.5)

8.35

(4)

4(0

.4)

Hon

g Ko

ng (S

AR)

814

.150

9(7

.4)

7.92

(10)

8(0

.8)

Net

herl

ands

814

.353

5(4

.7)

8.00

(7)

5(0

.8)

Swit

zerl

and

814

.752

6(4

.6)

7.78

(13)

7(0

.6)

Benc

hmar

king

par

tici

pant

s

New

foun

dlan

d an

d La

brad

or, C

anad

a8

13.8

528

(2.8

)7.

38 (2

0)²

6(0

.0)

Ont

ario

, Can

ada

813

.854

7(3

.2)

7.38

(20)

²6

(0.3

)

Benc

hmar

king

par

tici

pant

s no

t m

eeti

ng s

ampl

ing

requ

irem

ents

City

of B

ueno

s A

ires

, Arg

enti

na8

14.2

450

(8.6

)5.

36 (5

3)¹

33(9

.4)

▲ ▼

() S

tan

da

rd e

rro

rs a

pp

ea

r in

pa

ren

the

ses.

Be

cau

se r

esu

lts a

re r

ou

nd

ed

to

th

e n

ea

rest

wh

ole

nu

mb

er,

so

me

to

tals

ma

y a

pp

ea

r in

con

sist

en

t.

Ave

rage

s fo

r C

IL E

duca

tion

al s

yste

ms,

yea

rs o

f sc

hoo

ling,

ave

rage

age

, IC

T In

dex

and

perc

enti

le g

raph

Ach

ieve

me

nt

sig

nifi

can

tly h

igh

er

tha

n I

CIL

S a

vera

ge

Ach

ieve

me

nt

sig

nifi

can

tly lo

we

r th

an

IC

ILS

ave

rag

e

Stud

ent

- com

pute

r ra

tios

Year

s of

sc

hool

ing

Ave

rage

ag

e

Com

pute

r an

d In

form

atio

n Li

tera

cy S

core

Ave

rage

CIL

sc

ore

* IC

T D

eve

lop

me

nt

Ind

ex

Sco

re a

nd

co

un

try

ran

k d

ata

20

12

(S

ou

rce

: h

ttp

://w

ww

.itu

.int/

ITU

-D/ic

t/p

ub

lica

tion

s/id

i/in

de

x.h

tml [

27

/02

/14

])

10

02

00

30

04

00

50

06

00

70

0

100

200

300

400

500

600

700

10

02

00

30

04

00

50

06

00

70

0

5th

Pe

rce

ntil

es

of

Pe

rfo

rma

nce

25

th7

5th

95

th

Me

an

an

d C

on

fide

nce

Inte

rva

l

Belo

w L

1L3

L1

L2

L4

10

02

00

30

04

00

50

06

00

70

0

25

2.3 CIL Proficiency Levels

Fraillon et al. (2014) report on their analysis of the achievement data from ICILS 2013 to find out if the measurement data for different strands and aspects revealed that these were in fact belonging to different dimensional scales. The analysis resulted in an extremely high latent correlation (96%) between the students’ achievement scores on strands 1 and 2, indicating that it is appropriate to report CIL as a single achievement scale.

A more descriptive CIL achievement scale was developed in order to provide a more qualitatively meaningful understanding of the performance levels of students with different achievement scores. To establish such a scale, an  “item  map”  was produced to rank the items (a unit of analysis that derives a score associated with a question or task) from the least to the most difficult. The item map and student achievement data were further analyzed to establish proficiency levels with a width of 85 scale points and level boundaries at 407, 492, 576, and 661 scale points.

The scale is hierarchical in that a student located at a higher position on the scale will be able to successfully complete tasks up to that level of achievement. The scale is constructed so that students achieving a score at the lower boundary of a level should have answered about 50% of the items in that level correctly. For anyone with a score higher than the level boundary, he/she should have answered more than 50% of the items at their respective level. Table 2.3 describes what a typical student at each of the levels would be able to do in terms of Strand 1 and Strand 2 capabilities.

At Level 1, students are able to use a basic range of software commands to access files and complete routine layout tasks and basic editing of text. They are also able to recognize potential risks associated with misuse by unauthorized use of computers. Students working at a level below Level 1 are likely to require support and guidance to be able to create digital information products.

At Level 2, students are able to locate explicit information from given digital sources, have basic abilities to select and add content to information products with some rudimentary ability to follow layout conventions. They are also aware of the mechanisms to protect personal information and the consequential risks of public access to personal information. The key difference between Level 2 and higher levels   of  proficiency   is   in   students’   ability   to  work   autonomously   and   critically  when accessing and using information to create information products.

At Level 3, students are able to search for, select and work with information appropriately to edit and create information products. They are aware of potential biases, inaccuracy and unreliability of information sources, and are able to control the layout and design of their digital products.

26

Table 2.3 Descriptions  of  students’  typical  performance  capabilities  at  the  different  levels  

of the CIL scale Proficiency level & CIL

score

Performance related to Strand 1: collecting & managing information

Performance related to Strand 2: producing & exchanging

information

Level 4 (>661)

Able to: x Select the most relevant

information to use for communicative purposes.

x Evaluate the usefulness of information based on criteria associated with need.

x Evaluate the reliability of information based on its content and probable origin.

Able to: x Create and adapt information

products for specific audience and communicative purpose.

x Use appropriate software features to restructure and present information using appropriate conventions.

x Show awareness of intellectual property issues in using information on the Internet.

Level 3 (576-660)

Able to: x Independently use computers as

information gathering and management tools.

x Select the most appropriate information source for a specified purpose.

x Retrieve information from given electronic sources to answer concrete questions.

x Use conventionally recognized software commands to edit/add content and reformat information products.

Able to: x Recognize that the credibility of

web-based information can be influenced by the identity, expertise and motives of the creators of the information.

Level 2 (492-575)

Able to: x Use computers to complete basic

and explicit information gathering and management tasks.

x Locate explicit information from within given electronic sources.

Able to: x Make basic edits and add content to

existing information products per specific instructions.

x Create simple information products that show consistency of design and layout conventions.

x Demonstrate awareness of mechanisms to protect personal information and of consequences of public access to personal information.

Level 1 (407 – 491)

Able to: x Demonstrate a working knowledge

of computers as tools x Understand the consequences of

computers being accessed by multiple users.

Able to: x Perform basic communication tasks

and add simple content to information products using conventional software commands.

x Demonstrate familiarity with basic layout conventions of electronic documents.

27

Compared to those at lower levels of proficiency, students operating at Level

4 have a higher level of precision in information search and selection, and more

sophisticated control in layout and formatting to meet the specific communicative

purpose of the digital products. They also demonstrate awareness of the

commercial potential of information products and issues related to intellectual

property rights of electronically sourced information.

Based on the above description, it can be argued that those students operating

below Level 3 do not have adequate CIL proficiency to cope with the challenges

they face in working with information in their everyday life and work.

Just as a student's level of proficiency in CIL is reflected by his/her scaled CIL

score, the item difficulty of questions and tasks indicates the probability of the item

being successfully answered by students—the higher the item difficulty, the lower

the proportion of submitted answers being successful. Table 2.4 provides a

description of the questions and tasks that students operating at each proficiency

level were typically able to answer correctly at least 50% of the time. These

questions  and  tasks  hence  provide  a  more  vivid  illustration  of  students’  capabilities  at each level. It also provides a helpful reference for students, teachers and other

educators alike on the more challenging tasks that students need to be able to tackle

in order to move to the next level of CIL proficiency.

2.4  Hong  Kong  Students’  CIL  Proficiency  Levels  in  an  International Context

The CIL proficiency level of each ICILS student participant can be computed based

on his/her CIL score. In order to take advantage of the possibility of interpreting

the findings from the Hong Kong results in the international context, and without

burdening our readers with too much information, we have selected two other

participating countries that are economically developed and familiar to people in

Hong Kong for the purpose of comparison.

Figure 2.1 is a pie chart showing the percentages of students performing at

each of the four proficiency levels, and those that performed at below Level 1 in

Hong Kong, Australia and Korea. It can be seen that Level 2 is the proficiency level

with the highest percentage of students in all three systems. In fact, even for Czech

Republic which is the country with the highest average CIL score and the highest

percentage of students operating above Level 2, that percentage was only 37%. If

we take the perspective that Level 3 is the minimum CIL for adequate functioning

in the information world around us, then even for the highest performing countries,

there is a lot that needs to be done in terms  of  promoting  students’  CIL  proficiency.  In Hong Kong, the situation is even less promising, with only 3% achieving Level

4 and 23% achieving Level 3.

28

Table 2.4 Examples of ICILS test items mapped to the four CIL proficiency levels based on their item difficulty

CIL level*

Matched Strand 1 items: collecting & managing

information

Matched Strand 2 items: producing & exchanging information

Level 4

x Evaluate the reliability of information intended to promote a product on a commercial website.

x Select, from a large set of results returned by a search engine, a result that meets the specified search criteria

x Select relevant images from electronic sources to represent a three-stage process.

x Select from sources and adapt text for a presentation to suit a specified audience and purpose.

x Appropriate use of colour in a presentation suited to the communicative purpose.

x Suitable use of text layout and formatting in different elements in an information poster.

x Balanced layout of text and images for an information sheet.

x Recognize the difference between legal, technical, and social requirements when using images on a website.

Level 3

x Evaluate the reliability of information presented on a crowd-sourced website

x Select an appropriate website navigation structure for a given content;

x Use generic online mapping software to represent text information as a route map.

x Select relevant information according to given criteria to include in a website.

x Select and adapt some relevant information from given sources when creating a poster.

x Demonstrate control of image layout when creating a poster.

x Demonstrate control of color and contrast to support readability of a poster

x Demonstrate control of text layout when creating a presentation

x Aware that a generic email greeting indicates sender  does  not  know  the  recipient’s  identity

Level 2

x Navigate to a URL presented as plain text.

x Locate simple information within a multi-page website.

x Add contacts to a collaborative workspace. x Insert information to a specified cell in a

spreadsheet. x Differentiate between paid and organic search

results returned by a search engine. x Use formatting and location to denote text with

the role of title in an information sheet. x Use the full page when laying out a poster. x Demonstrate basic control of text layout and color

use when creating a presentation. x Explain a potential problem if a personal email

address is publicly available.

Level 1 x Open a link in a new

browser tab. x Insert an image into a

document

x Use software to crop an image. x Place a title prominently on a webpage. x Create a suitable title for a presentation. x Identify email recipients on carbon copy list. x Name risk(s) in not logging out after using a

publicly accessible computer.

29

* There were a few items that had scaled difficulties below Level 1. These only involve execution of the most basic skills, such as clicking on a hyperlink.

Figure 2.1 Percentages of Hong Kong, Australia and Korea students performing at each CIL proficiency level

An even more worrisome picture emerges if we look at the percentages of students that were assessed to perform at below Level 1. There were 15% of students in Hong Kong who were assessed to perform at below Level 1. Only four systems had a higher percentage of students operating at below Level 1: Turkey, Thailand, City of Buenos Aires (Argentina) and Chile. Clearly, there is a serious mismatch  between  our  students’  CIL  proficiency  level  and  Hong  Kong’s  overall  ICT development index as 10th in the world and 5th among the 21 participating systems. Clearly, improving the CIL proficiency level is one of the critical educational priorities for Hong Kong if we are to have a workforce and a citizenry that are CIL proficient for life and work in a highly digitized world. In the next section,  we  are  going  to  explore  the  students’  strengths  and  weakness  across  the  seven Aspects within the two CIL Strands at each of the seven CIL levels in order to seek a better understanding of the possible trajectory of development in CIL as students’ progress up the various levels.

30

2.5 Students’  Performance across CIL Aspects

As explained in section 1.4, the CIL assessment framework in this Study has two strands, which can be further differentiated into a total of seven aspects. Each of the items in the assessment was designed to assess one of the seven aspects. Hence, it is possible to compute the mean percentage correct for each student for each set of items in a particular aspect and from these percentage scores to compute the mean performance of the entire population of students for each CIL aspect. Table 2.5 presents the percentage of correct answers and the relative difficulty for each aspect for students in Hong Kong, Australia and Korea. The results clearly show that while these three systems differ greatly in the students’  overall performance, they all find accessing and evaluating information, and transforming information to be the most difficult; whereas items related to knowing about and understanding computer use and using information safely and securely were the easiest. Table 2.5 The percentages correct (S.D.) and relative difficulty for the seven CIL aspects for Hong Kong, Australia and Korea

CIL strand & aspect HK AUS KOR % (S.D.) D* % (S.D.) D* % (S.D.) D*

Strand 1: Collecting and managing information 1.1 Knowing about and understanding

computer use 71 (23) 7 76 (23) 7 74 (22) 7

1.2 Accessing and evaluating information 38 (31) 2 42 (31) 1 40 (31) 1

1.3 Managing information 43 (43) 4 45 (43) 3 49 (41) 5

Strand 2: Producing and exchanging information

2.1 Transforming information 36 (25) 1 48 (26) 2 46 (27) 2

2.2 Creating information 53 (28) 5 59 (24) 5 56 (27) 4

2.3 Sharing information 39 (37) 3 49 (35) 4 44 (38) 3 2.4 Using information safely and

securely 58 (25) 6 61 (25) 6 62 (22) 6

D* is the relative difficulty of the aspect in comparison with the others. 1 is most difficult. In the remainder of this section, we will report on the common errors and

difficulties that Hong Kong students had in completing the tasks in each of the CIL aspects through the analysis of  the  students’  answers to the tasks and questions in the released module, After-school Exercise.

31

2.5.1 Knowing about and understanding computer use (Aspect 1.1)

Items related to this aspect were found to be the least difficult by Hong Kong and the two reference countries, Australia and Korea. Test items for this aspect are mainly multiple choice or action items in the short tasks section of the student test modules. In the After-school Exercise module, tasks under this aspect required students to know how to navigate to another webpage when given different forms of instructions. Nearly all students (93%) could open a web document when given the hyperlink next to the instruction (Figure 2.2a), and 80% could do the same when given a hyperlink placed inside a pop-up bubble (Figure 2.2b). However, only 65% of the students were able to copy and paste a non-hyperlinked URL to the browser task bar to navigate to the designated page (Figure 2.2c).

Figure 2.2 The  three  tasks  that  test  students’  ability  to  navigate  to  a  designated  webpage  using different forms of instruction

2.5.2 Accessing and evaluating information (Aspect 1.2)

This was the second most difficult aspect for Hong Kong students and the most difficult for both Australian and Korean students. In the After-school Exercise module, this aspect was assessed within the large task, under the criterion information completeness, and was applied to assess the text on the poster created by the students.

a. Hyperlink as circled. b. Hyperlink placed inside circled bubble.

c. Student has to copy and paste circled URL to browser task bar.

32

In order to get the full credit for two score points, students have to put down all three pieces of information related to the After-School Exercise program: when the event will take place (both days and time), what people will do, and what equipment and/or clothing will be needed, as explicitly specified in the task criteria presented at the beginning of the large task (see Figure 1.15). Only 16% of the Hong Kong students succeeded in putting all three pieces of information on their poster, which is very low compared to Australian (40%) and Korean (31%) students. A further 35% of the Hong Kong students were able to provide two out of the three pieces of information to score a partial credit of one point. The poster in Figure 2.3 is an example of a poster that has received a full credit of 2 score points on information completeness, while the one in figure 2.4 did not receive any credit at all as there was no relevant information that matched what was required.

Figure 2.3 A poster designed by a Hong Kong student. (Full credit on information completeness)

Figure 2.4 Sample poster designed by Hong Kong student (0 score information completeness)

2.5.3 Managing information (Aspect 1.3)

Managing information was found to be of medium difficulty to students in Hong Kong as well as in Australia and Korea. On the other hand, this is the aspect that has the largest standard deviation for the mean for Hong Kong, indicating a large diversity in students’  ability  to  tackle  this  task.  In After-school Exercise, this aspect was evaluated through a short task in which students were asked to grant a co-learner the right to edit an online document by changing its sharing setting to “to  edit”. Figure 2.5 shows the screen layout for this task. Only 50% of Hong Kong students were able to change the settings correctly, compared to 72% of Australian and 66% of Korean students.

33

Figure 2.5 Screenshot of the short task in After-school  Exercise  that  tests  students’  ability  to change the sharing setting of a web document

2.5.4 Transforming information (Aspect 2.1)

As evident from Table 2.5, transforming information was found to be the most difficult CIL aspect by Hong Kong students, and the second most difficult by Australian and Korean students. In the After-school Exercise module, this was assessed via four separate criteria in the poster design task: title design, colour contrast, persuasiveness of the poster and use of the full page. Title Design: The title design was evaluated against two performance expectations (see Table 1.5), one of which relates to this aspect: whether a relevant title has been added and formatted to make its role clear. This requires the title to be placed in a prominent position and the text formatting to make it distinctive by capitalizing, using larger font size, bolding, etc. The posters in Figures 2.3, 2.4 and 2.6 have met this criterion. In Figure 2.7, even though   the   title   “FENCING”  was   capitalized,  bolded and underlined, it was not placed in the middle in a sufficiently prominent position to signal that it was a title, and hence failed to meet this criterion in the assessment. In summary, 49% of Hong Kong students submitted posters that met this criterion, compared to 64% of Australian and 50% of Korean students respectively.

34

Figure 2.6 Sample poster designed by a

Hong Kong student

Figure 2.7 Sample poster designed by a

Hong Kong student

Colour Contrast: Colour contrast was the second criteria associated with the CIL

aspect transforming information, which essentially assesses whether there is

sufficient contrast between the text and background of the poster for easy reading.

To meet this criterion, students need to have some idea about colour schemes and

how  these  can  be  used  to  direct  readers’  attention  to  important  details  on  the  poster.  This criterion was machine scored. The colour contrast of the posters in Figures 2.3,

2.4 and 2.6 were all considered to be adequate to achieve this score point, even

though the contrasts may differ across these three posters. In comparison, the

colour contrast for the poster in Figure 2.7 was poor. Only 11% of Hong Kong

students submitted posters that met the criteria for good colour contrast while the

respective percentages for Australia and Korea were 25% and 16%.

Persuasiveness of Poster: This criterion considers whether the poster includes

persuasive or emotive text (which can be placed anywhere on the poster) to attract

people to participate in the programme. The posters in Figures 2.3, 2.6 and 2.7 have

met this criterion. The relevant texts for this criterion are:

Figure 2.3: (within the main text) Finding yourself performed not to well in the PE

examination? Want to increase your performance in the PE examination? Want to keep fit?

Then join this Marvellous Programme!

Figure 2.6: (in the title) 30 分鐘運動好, 健康生活齊做到 (translation: 30 minutes

exercise is good, we can all live a healthy life)

Figure 2.7: (at the bottom of the poster) Isn't it interesting? Come to join us!

35

Only 10% of Hong Kong students received the score point for the

persuasiveness criterion, as compared to 38% of Australian and 60% of Korean

students respectively.

Use of full page: This is a relative simple criteria, assessed by checking whether the

students have made use of the full area of the poster to display the information.

Half of the students in Hong Kong (50%) met this criterion, while in Australia and

Korea the corresponding figures were 61% and 57% respectively. Figure 2.3 and

Figure 2.4 did not received any credit in this area.

Overall, it can be seen that even though transforming information is also a

relatively difficult CIL aspect for Australia and Korea, Hong Kong students are

achieving a much lower level compared to their Australian and Korean

counterparts.

2.5.5 Creating information (Aspect 2.2)

Creating information was a comparatively well-answered CIL aspect for Hong

Kong students. It was assessed through four of the poster design criteria: title

design, image layout, text layout and formatting, and colour contrast. As can be

seen from Table 1.5, two of these criteria, title design and colour contrast, also

applied to the transforming information aspect. In both cases, the level of

performance for achieving the creating information criterion is lower than that for

transforming information, as will be explained below.

Title Design: A poster will be assessed to have achieved the creating information

criterion if a relevant title has been added and placed in a prominent position.

Hence, satisfying this criterion for the title design is a necessary prerequisite for it

to be further considered against the criterion for transforming information. Thus,

the titles in the posters in Figures 2.3, 2.4 and 2.6 all satisfy the creating information

criterion, and the title in Figure 2.7 does not. The percentages of students meeting

this criterion for Hong Kong, Australia and Korea are respectively 69%, 80% and

71%.

Image layout: This criterion requires that one or more images are well aligned with

the other elements on the poster and appropriately sized. The posters in Figure 2.4

and 2.7 do not contain images and do not satisfy this criterion. The images in Figure

2.3 are well laid out and aligned with the meaning of the text in the poster, which

thus gained the full score for this criterion. In comparison, the images in Figure 2.6

overlap with the main text on the poster, and did not satisfy this criterion. Only

32% of Hong Kong students submitted posters that satisfied this criterion, while

the corresponding performance of Australian and Korean students were 50% and

49% respectively.

36

Text layout and formatting: This criterion concerns whether the students have used formatting tools to show the role of different text elements on the poster. The assessment differentiates between two levels of performance. At the lower level, a student needs to demonstrate basic control of text layout on the poster. A higher level of performance requires that a consistent layout format has been used to show the roles of different text elements. Hence, the poster in Figure 2.4 only received one score point (assessed to be at a lower level of performance) as the formatting was relatively basic. On the other hand, the poster in Figure 2.3 was able to demonstrate the use of different formatting features for different text elements, and received a full score of two points. Only 11% of Hong Kong students received the full score on this criterion, compared to the corresponding figures of 19% and 27% by Australian and Korean students respectively. Another 41% of the students in Hong Kong were assessed to perform at the lower level for this criterion, while the corresponding figures for Australia and Korea were both 46%.

2.5.6 Sharing information (Aspect 2.3)

In the After-school Exercise module, sharing information was assessed through one short task and two poster design assessment criteria, colour consistency and information adaptation. For the short task, the students were simply assessed on whether they could identify who received an email via carbon copy. This item was relatively well answered by Hong Kong students with 69% correct, while the respective figures for Australia and Korea were 80% and 57% respectively. Colour consistency: This criterion evaluates whether a poster shows evidence of planning regarding the use of colour to denote the role of text, background and images on the poster. The default setting of the poster creation software was such that if the students did not at least change either the background colour or the text colour, the text would be very difficult to read because of poor contrast. The scoring follows a very simple principle: colour should be used intentionally to suit the purpose of the text within the poster, and not just randomly or decoratively. It is not necessary to use multiple colours to receive credit for this criterion as readability is the primary concern. All four posters shown in Figures 2.3, 2.4, 2.6 and 2.7 received full score for this criterion. A total of 76% of Hong Kong students met this criterion, compared to 67% of Australian and 76% of Korean students. Information adaptation: This criterion assesses the extent to which useful information has  been  identified  and  suitably  presented  using  the  student’s  own  words  on  the  poster. No credit would be awarded if chunks of text were directly copied from the given resources and pasted with no or very minimal editing. One score point would be awarded if some editing were done to improve ease of comprehension and relevance. Full score (two points) would be awarded if the student rephrased

37

all the key points on the poster using his/her own words. Information adaptation is the weakest area for Hong Kong students, with only 8% gaining at least one score point (i.e. having done some level of editing), and a dismal 1% scoring two score points by doing substantial rephrasing. This stands in stark contrast with Australia, which has corresponding figures of 52% and 13%, and Korea with 63% and 33% respectively.

None of the four posters shown earlier scored even one point on this criterion. Figure 2.3 does not provide any detail about either of the sports listed on the poster. On the other three posters (Figures 2.4, 2.6, and 2.7), all the text about the sports were copied directly from the given resources and pasted with no editing.

2.5.7 Using information safely and securely (Aspect 2.4)

This CIL aspect was assessed through five short tasks in the After-school Exercise module. The tasks and students’   responses are described below. Risks in revealing personal information: In short task 3, students were asked which of four kinds of personal information would be the most risky to make   public   in   one’s   profile information when opening an account on a public website: name, gender, home address and nationality (see Figure 2.8). For this item, Hong Kong students had excellent performance, with  96%  being  able   to  choose  “home  address”   as   the   correct   answer.   This  compares very well with the corresponding performance of Australian and Korean students, at 95% and 90% respectively.

Detecting fraudulent (phishing) emails: The short tasks 5, 6 and 7 all referred to the email in Figure 2.9, and ask the student to point out why each of the three elements A, B and C (denoting two URLs, C1 and C2) suggest that this is a phishing email. Samples  of  Hong  Kong  students’  answers  are  presented  in  Table  2.6.

Figure 2.8 The short task that assessed students’  awareness  of  risks  in  making personal information public

38

Table 2.6 Sample responses from Hong Kong students to the three tasks related to the phishing email

Responses  scoring  “0” Responses  scoring  “1”

Task 5: suspicious sender email

Can only say that the email is suspicious, e.g.: x I  don’t  know  the  sender x Fake email x Not an official website

Can explain that the email did not originate from Webdocs, e.g.: x Email is not sent by Webdocs x This is not commercial/

professional email x There  is  the  word  “freemail”.  

Security would not use this word

Task 6: suspicious greeting

Unclear answers, such as: x Should  not  use  “Dear” x There should not be a yellow

highlight x My name is not WEBDOCS

Can clearly state that the greeting does not have the name of the client, e.g.: x If real, should use name: XXX x Should  use  customer’s  name

Task 7: suspicious URL

Unclear answers, such as: x No www x No  “.com”  and  no  “.hk” x Has  “reset”

Can identify the problem, e.g.: x Not sure that the reset password

website belong to the Webdocs company

x The URL links to freeweb x They are different websites

A  is  the  email  sender’s  address.  The  student  should  point  out  that  the  domain  

name   of   the   purported   sender’s   address   does   not   correspond   to   that   of   the  purported  sending  organization’s  domain.  24%  of  the  Hong  Kong  students  were  able to answer this item correctly, compared to 19% and 21% of Australian and Korean students respectively.

B is the greeting to the person receiving this email. If this email actually came from the purported organization, it should have the name of the receiver and so the greeting should be personal. In Hong Kong, only 24% of the students were able to point out that the greeting in B was not a personal one. The percentage of Korean students who answered this question correctly was similar to that of Hong Kong, at 27%, while Australian students performed much better, with 60% correct.

39

Figure 2.9 The phishing email with three suspicious elements, A, B and C The third short task related to this phishing email asks students to identify

why the two URLs identified by C may be suspicious, and they should point out that the explicit URL address C1 is not the same as the URL of the actual hyperlink as  indicated  by  C2.  Hong  Kong  students’  performance  in  this  item  is  similarly  poor  as in the previous two, with only 24% correct. On the other hand, this item proves to be the most difficult out of the three on this phishing email for Australian students, with only 15% getting the item correct, while the performance of Korean students was similar to that of Hong Kong, with 23% correct. Problem  of  making  one’s  email  address  public: The final short task in this module that assesses  the  CIL  aspect  “using  information  safely  and  securely”  asks  the  students  to  name  one  problem  that  may  arise   if  one’s  email  address   is  made  public.  Any response indicating either that this may result in getting unwanted email such as spam and advertising email, or that this may lead to loss of privacy such as being contacted by strangers or stalkers would be acceptable answers. However, some students   also   mistakenly   thought   that   this   might   lead   to   one’s   account   being  hacked,   or   one’s   information   being   stolen. Samples   of   students’   incorrect   and  correct responses are presented in Table 2.7.

40

Table 2.7 Samples  of  Hong  Kong  students’  responses  to  the  short  task  on  problems  that  

may  arise  by  making  one’s  email  address  public Responses scoring  “0” Responses  scoring  “1”

This item was of medium difficulty for Hong Kong students, with 45% of the

responses scored as correct, while the Australian and Korean students found it to

be relatively easy, with correct percentages at 62% and 78% respectively.

2.5.8  Students’  performance  across  CIL  aspects  and  proficiency  levels

As mentioned in section 2.4, all of the items in the CIL assessment instruments were

scaled and mapped onto four CIL proficiency levels. Tables 2.8 and 2.9 summarize

the performance of students from Hong Kong, Australia and Korea for each of the

items mapped onto the CIL aspects and levels. In general, fewer students will be

able to answer satisfactorily items that are mapped at a higher CIL level.

Overall, it  can  be  seen  that  while  the  performance  tasks  assess  students’  CIL,  language competence also matter, particularly for items that require open-ended

written response. Hong Kong students tend to be weak at tasks that require the use

of language to express their ideas clearly in their own words, such as information

adaptation and questions that require explanation.

41

Table 2.8 Percentage of students who have correctly answered each short task in the After-school Exercise module (mapped to CIL assessment aspect and level)

CIL aspect Task no. CIL level

Percentages of students correct

HK AUS KOR

1.1 Knowing about and

understanding computer use

2 Navigate to a URL given as plain text 2 65 66 63

4 Open webmail from hyperlink in pop up bubble 1 80 85 91

9 Access a document from hyperlink in email message 1 93 96 97

1.3 Managing information 10 Modify the sharing settings

of a collaborative document 2 50 72 66

2.3 Sharing information 1 Identify who received an

email by carbon copy 1 69 80 57

2.4 Using information safely and securely

3 Identify personal information risky to set as public in profile.

2 96 95 90

5 Identify sender email & organization domain mismatch as suspicious

4 24 19 21

6 Identify a generic email greeting to be suspicious 3 24 60 27

7 Identify difference in URL displayed & the hyperlink address as suspicious

4 24 15 23

8 Explain potential problem in making personal email address public

2 45 62 78

Another observation is that while the difficulty in completing a task

satisfactorily depends on the specific CIL aspect it assesses, the format and context of the task as well as the performance level required also matter. Hence the students’   performance   in   the   various   tasks   and  CIL   aspects   in   the  After-school Exercise   may   not   necessarily   provide   a   good   reflection   on   the   students’   CIL  achievement.  We  could  get  a  better  estimate  of   the  students’  performance if we make use the performance data collected from all four assessment modules. This will be reported in the next section.

42

Table 2.9 Percentage of students who have achieved partial or full score for each assessment criterion (mapped to CIL level) in the poster design large-task

Criterion Score/ max. score CIL aspect CIL level

Percentages of students correct

HK AUS KOR

1. Title design 1/2 2.2 Creating information 2 69 80 71

2/2 2.1 Transforming information 2 49 64 50

2. Image layout 1/1 2.2 Creating information 3 32 50 49

3. Text layout & formatting

1/2 2.2 Creating information 2 52 65 73

2/2 2.2 Creating information 4 11 19 27

4. Color contrast 1/2 2.2 Creating information 1 66 76 73

2/2 2.1 Transforming information 3 11 25 16

5. Color consistency 1/1 2.3 Sharing information 1 76 67 76

6. Information adaptation

1/2 2.3. Sharing information 3 8 52 63

2/2 2.3. Sharing information 4 1 13 33

7. Information completeness

1/2 1.2. Accessing & eval. information 2 51 65 63

2/2 1.2. Accessing & eval. information 3 16 40 31

8. Persuasiveness 1/1 2.1. Transforming information 3 10 38 60

9. Use of full page 1/1 2.1. Transforming information 2 50 61 57

2.6 Students’  CIL  proficiency  trajectories

From a pedagogical point of view, it would be useful to know the strengths and weaknesses  of  a  student’s  CIL  beyond  his/her  performance at an item level. In this section,   we   will   explore   the   relative   strengths   and   weaknesses   of   students’  performance in each of the seven aspects, and to understand how students at various proficiency levels differ in terms of their achievements in each of the seven aspects. Such knowledge would help to provide some initial information about students’  trajectory  of  improvement across the CIL aspects, which may be used to guide teachers in designing programs of CIL learning for students. We will also compare   such   “trajectories”   for   Hong   Kong,   Australia   and   Korea   to   examine  whether such trajectories are generally applicable or whether these are actually dependent on the educational and cultural context of individual countries. Figure 2.10 is a graphic representation of the percentages correct for each of the seven CIL aspects for Hong Kong, Australian and Korean students presented in Table 2.5.

43

Figure 2.10 A radar diagram of the mean percentages correct per CIL aspect

The key areas of concern in terms of the low percentages correct (below 40%)

are aspects 1.2 (Accessing and evaluating information), 2.1 (transforming information), and 2.3 (sharing information). It can also be seen that while the two comparison countries are  both  high  performing  systems   in   ICILS,  Hong  Kong’s  achievement in some of the aspects are not too far behind. The key areas that Hong Kong students lag behind are aspects 2.1 (transforming information), 2.3 (sharing information) and 1.3 (managing information). In the remainder of this section, we will explore how the profile of competence differs across the five groups of students from below Level 1 to Level 4. These profiles will help us understand how students’  CIL  competence  develops from the lowest to the highest level.

44

2.6.1  Hong  Kong  students’  improvements  in  competency  profile  as  they  move to higher CIL proficiency levels

Figure 2.11 is the graphic representation of the profiles for Hong Kong students at

each of the five CIL proficiency levels. The group of students at Below Level 1

(comprising 15% of the Hong Kong student population) can be considered as the

least  CIL  “educated”  group  of  students.  It  can  be  seen  that  even  for  this  group,  CIL  aspects 1.1 and 2.4 are relatively well developed.

Figure 2.11 A radar diagram showing the mean percentages correct per CIL aspect for Hong Kong students at each of the five CIL proficiency levels

Since Hong Kong students have extremely high computer (98%) and Internet

(99%) access at home, and there is pervasive use of Internet technology in the

society at large for social communications and entertainment, the relatively high

achievement in these two aspects of CIL competence are probably unrelated to

students’   educational experiences in schools. Comparing the CIL competency

profile between this group and the Level 1 group, we can see that there is a marked

increase in competency for the latter in all seven CIL aspects.

45

As Fraillon et al. (2014) mentioned, students performing below Level 1 were in fact outside of the competence level that the ICILS assessment was designed for. Clearly, the fact that we still have such a large proportion of students in Hong Kong operating at such a low level is very worrying. Further, there is a need to find ways to help this group of students comprehensively in all 7 CIL aspects in order to reach at least Level 1 proficiency.

The developmental picture becomes rather different when we compare the competency profiles beyond Level 1. From Level 1 to Level 4, the improvements in aspects 1.1 (knowing about and understanding computer use) and 2.4 (using information safely and securely) were much smaller in between any two adjacent levels. In two of the other five aspects, 1.2 (accessing and evaluating information) and 2.1 (transforming information), there continued to be marked improvements in  students’  competence  going  up  the  proficiency  levels.

However, for the remaining three CIL aspects 1.3 (managing information), 2.2 (creating information), and 2.3 (sharing information), there were close to no improvement between Level 3 to Level 4. This means that our highest performing students were not able to advance further in competence in these three CIL aspects. This is particularly worrying for sharing information (aspect 2.3) and managing information (aspect 1.3) for which the highest mean for Level 4 students were below 60% and below 70% respectively.

2.6.2 Australian and Korean  students’  improvements  in  competency profile as they move to higher CIL proficiency levels

It is useful to find out if other countries show similar developmental patterns in CIL competency as students advance in their CIL proficiency levels. Figures 2.12 and 2.13 show the corresponding developmental profiles for Australian and Korean students respectively.

Comparing the three Figures 2.11 to 2.13, there is clear resemblance in profile across the three systems among students at Below Level 1 proficiency, and that the improvement from Below Level 1 to Level 1 are similarly marked in all seven aspects.

On the other hand, Australia appear to have the most balanced improvement in all seven CIL aspects such that their students at Level 4 achieve a minimum mean percentage correct of 70% for the lowest performing aspect, sharing information (aspect 2.3); and except for managing information (aspect 1.3), this group of highest performing students achieve a mean of at least 80% correct in all the other five aspects.

46

Figure 2.12 A radar diagram showing the mean percentages correct per CIL aspect for Australian students at each of the five CIL proficiency levels

Korean students also show a more balanced improvement profile compared to Hong Kong, though less so than Australia. Korean students at proficiency level 4 also achieved a minimum mean percentage correct above 70% in all seven aspects.

47

Figure 2.13 A radar diagram showing the mean percentages correct per CIL aspect for Korean students at each of the five CIL proficiency levels

2.7 Summary

Results from the ICILS 2013 study show that Hong Kong students had lower

achievement in Computer and Information Literacy compared to all the

economically developed countries participating in the study as measured by the

overall CIL score. Based on the CIL proficiency levels as defined by the Study, 15%

of Hong Kong students did not even reach Level 1 proficiency, which was the

lowest level that the Study was designed to assess. In fact, based on the expected

competencies at the different levels of proficiency (see Table 2.3), a person with CIL

proficiency below level 3 is seriously handicapped in his/her CIL ability to cope

with the everyday demands in the information world around us. Only 26% of the

students in Hong Kong achieved Level 3 or above while the corresponding

percentages were 34% and 35% respectively for Australia and Korea.

There is clearly a serious mismatch between the  students’  CIL  proficiency  and  the role played by computer and information technology in all aspects of social,

economic and political developments in Hong Kong.

48

A   more   refined   analysis   reveals   important   differences   in   students’  achievement across the seven CIL aspects in the assessment framework. Hong Kong students had the poorest performance in accessing and evaluating information, transforming information and sharing information, with a mean of lower than 40% correct in each of these three aspects, and in managing information (43% correct). Through comparing  students’   competence  profiles  across   the   five  proficiency levels from Below Level 1 to Level 4, it was found that Hong Kong students were not able to show similar advancement in all seven CIL aspects, whereas their Australian and Korean counterparts showed much more balanced improvement in all seven aspects. In particular, there was no significant advancement between Levels 3 and 4 in two of the lowest performing aspects, managing information and sharing information. This indicates that even for the highest performing students in Hong Kong, their learning experiences within and outside of the school did not help them to improve in critical CIL aspects. Clearly, these are issues of serious concern. In chapters 3 and 4, we will further explore the school level conditions and personal background characteristics that influence students’  CIL  achievement.

49

Chapter 3

Influence of Students’ Background and ICT Use Experience on their CIL

Students’  CIL  achievement   is   influenced  by  hierarchically  embedded  contextual  factors at multiple levels, namely, the personal level (e.g. ability, interest), family level (e.g. socio- economic status (SES) factors such as parental occupation and education),   school   level   (e.g.   teachers’   qualifications)   and   system   level   (e.g.  curriculum). The factors at each level can be further categorized into antecedents or  processes.  Process  factors  are  those  that  directly  influence  students’  learning in CIL, such as their opportunities to use ICT at home and in school, and their teachers’  pedagogical  ICT  competences.  Antecedent  factors  are  those  that  influence  the   process   factors.   Examples   of   antecedents   include   students’   family   SES,   the  school’s   student   intake   and   provisions   for   teachers’   professional   development.  Data pertaining to antecedent and process factors were collected through two approaches. The first is through the surveys of students, teachers, principals and ICT coordinators. The second is through a national context survey completed by each national research centre. These survey data allow us to investigate how factors at  different  levels  may  affect  students’  CIL  achievement.

3.1 Contextual Factors Derived from the Student Survey

Figure 3.1 depicts the conceptual framework adopted in ICILS 2013, based on which the context questionnaires were designed. This chapter reports on the findings related to the contextual data gathered through the student survey. All participating students, after completing the one-hour CIL performance assessment, were invited to take part in a 20-minutes online survey.

This   survey  was  designed   to   collect   information  on   the   students’  personal  and family background, as well as their use of and engagement with ICT at home and in school.

50

(Fraillon et al 2014, p.37) Figure 3.1 Contextual  factors  influencing  students’  CIL  outcomes

Table 3.1 lists the set of 13 variables derived from the student survey. Six of

these are antecedents (S_SEX, S_ISCED, S_NISB, S_BASEFF, S_ADVEFF,

S_INTRST), while the remaining seven can be considered as process indicators

(S_TSKLRN, S_USEAPP, S_USELRN, S_USESTD, S_USEREC, S_USECOM and

S_USEINF). Conceptually speaking, one may differentiate between antecedent and

process factors, but actually they mutually influence each other. For example, more

opportunities  to  learn  to  use  ICT  may  help  to  develop  students’  stronger  interest  and self-efficacy. In this chapter, we first report on the basic descriptive statistics

of  these  context  variables  and  how  they  relate  to  the  students’  CIL  scores.  This  will  be followed by multilevel analysis to explore how these factors together influence

the overall CIL score and the standardized scores of each of the seven CIL aspects

of a student.

Data  collected  through  the  student  survey  reflects  the  students’  experiences,  and the classroom and school level conditions as perceived by the students. It is

important to note that variables derived from data collected through the student

survey may also reflect  conditions  at  other  levels.  For  example,  students’  reported  ICT  usage  in  learning  at  school  may  be  considered  as  a  reflection  of  their  teachers’  pedagogical use of ICT. However, for simplicity, we will initially treat all these

variables/factors as student level factors.

Also, we have limited our cross-national comparisons to only two countries,

Australia and Korea, so as to provide a sharper focus on systems that are of

stronger interest and familiarity to readers in Hong Kong.

Reports on the entire set of results from all of the 21 participating countries

can be found in Preparing for life in a digital age: The IEA International Computer and Information Literacy Study international report (Fraillon et al., 2014).

51

Table 3.1 List of key context variables* derived from the student questionnaire Context variable Description

S_SEX Gender of the student

S_ISCED Students’  own  expected  highest  level  of  education  reached

S_NISB National  index  of  students’  socioeconomic  background

S_BASEFF Self-efficacy in basic ICT skills (e.g. edit documents/photos, filing)

S_ADVEFF Self-efficacy in advanced ICT skills (e.g. handle viruses, create macro)

S_INTRST Interest and enjoyment in using ICT

S_TSKLRN Learning of CIL-related tasks at school (e.g. accessing, organizing and presenting information)

S_USEAPP Use of work-oriented ICT applications (incl. Office suite, programming, graphics and academic software)

S_USELRN Use of ICT during lessons at school in major, non-ICT school subjects

S_USESTD Use of ICT for study purposes (e.g. prepare assignments, collaborate)

S_USEREC Use of ICT for recreation (e.g. music, news, video, reading reviews)

S_USECOM Use of ICT for social communication (incl. texting & social media)

S_USEINF Use of ICT for exchanging information (e.g. posting on forums, blogs)

* Note: All except the first two variables are scaled indices derived from multiple item responses in the survey. For details of the construction of these scales, see Fraillon et al (2015) pp. 187-197.

3.2  Influence  of  Students’  Personal  and  Family  Background

There were two questions in the student survey that collected personal background variables: gender of the student, and the highest level of education that the student expected himself/herself to reach. There were four kinds of family background variables elicited by the survey: whether the student has recent immigrant status, language spoken at home with respect to the language used in the CIL assessment, socioeconomic status (SES) and the availability of ICT resources at home. Findings related to these variables are reported in this section.

52

3.2.1 Gender and CIL achievement

In past IEA studies, gender differences in reading literacy were mostly in favour of female students while achievements in science and math were more commonly in favour of male students, though such trends are also changing and vary across countries. From the results of ICILS 2013, we find that girls perform better than boys in all participating countries, and such differences were significant in all but four countries. Table 3.2 presents the gender differences in the mean CIL score for Hong Kong, Australia and Korea against the international average. It shows that female students achieved an average of 25 score points higher than their male counterparts. This difference is higher than the international average, which is only an 18 point differences, but similar to that found in Australia. Table 3.2 Gender differences in CIL

3.2.2 Educational aspirations and CIL achievement

One of the questions in the student survey asked the student to indicate the highest level of educational qualification he/she expects to attain. Usually, students with higher academic achievement in school tend to have higher educational aspirations. Hence it is not surprising that in all participating countries, students with higher educational aspirations achieve significantly higher CIL scores. However, it is worth noting that this difference is lowest in Hong Kong. Table 3.3 shows the results for Hong Kong in comparison with the international mean and the two comparison countries.

Hong Kong (SAR) 498 (9.2) 523 (7.5) 25 (8.3)

ICILS 2013 average 491 (1.0) 509 (1.0) 18 (1.0)

Australia 529 (3.3) 554 (2.8) 24 (4.0)

Republic of Korea 517 (3.7) 556 (3.1) 38 (4.1)

* Statistically significant (p<0.05) coefficients in bold.

Gender difference

Gender

Educational systemsDifference(Females -

Males)

() Standard errors appear in parentheses. Because some results are rounded to the nearest w hole number, some totals may appear inconsistent.

MalesMean Scale

Score

FemalesMean Scale

Score 0 25 50

FemalesScoreHigher

Gender difference statistically

Gender difference not statistically signif icant.

53

Hon

g K

ong

(SA

R)

14(1

.2)

479

(13.

8)8

(0.8

)45

4(1

0.6)

15(0

.8)

494

(9.9

)63

(1.8

)52

8(5

.9)

48(1

1.1)

ICIL

S 2

013

aver

age

8(0

.2)

439

(2.6

)24

(0.3

)46

6(1

.4)

17(0

.2)

493

(1.4

)51

(0.4

)52

7(0

.9)

89(2

.6)

Aus

tralia

6(0

.4)

463

(7.0

)18

(0.9

)50

7(3

.7)

16(0

.7)

524

(3.2

)60

(1.2

)56

6(2

.4)

103

(7.3

)

Kor

ea, R

ep. o

f4

(0.4

)47

2(9

.8)

9(0

.7)

489

(7.0

)13

(0.7

)52

2(5

.2)

74(1

.0)

548

(2.9

)76

(9.8

)

* S

tatis

tical

ly s

igni

fican

t (p<

0.05

) coe

ffici

ents

in b

old.

Uppe

r-sec

onda

ry

educ

atio

nPo

st-s

econ

dary

non

-un

iver

sity

edu

catio

nTe

rtiar

y un

iver

sity

ed

ucat

ion

Scor

e po

int d

iffer

ence

be

twee

n lo

wes

t and

hig

hest

ca

tego

ry

Educ

atio

nal s

yste

ms

CIL

scor

e di

ffere

nce

(Ter

tiary

ed

ucat

ion

- lo

wer

se

cond

ary

or

belo

w)

Stud

ents

' exp

ecte

d ed

ucat

ion

leve

l

Low

er-s

econ

dary

ed

ucat

ion

or lo

wer

() S

tand

ard

erro

rs a

ppea

r in

pare

nthe

ses.

Bec

ause

som

e re

sults

are

roun

ded

to th

e ne

ares

t who

le n

umbe

r, so

me

tota

ls m

ay a

ppea

r in

cons

iste

nt.

Mea

n CI

L sc

ore

%M

ean

CIL

scor

e%

Mea

n CI

L sc

ore

%M

ean

CIL

scor

e%

025

5075

100

125

Diff

eren

ce s

tatis

tical

ly si

gnifi

cant

at .

05 le

vel.

Diff

eren

ce n

otst

atis

tical

ly si

gnifi

cant

.

Stud

ents

in

hi

ghes

tca

tego

rysc

ore

high

er

than

in

lowe

st

Tabl

e 3.3

The

per

cent

ages

of s

tude

nts a

t eac

h le

vel o

f edu

catio

nal a

spir

atio

n an

d th

eir r

espe

ctiv

e CIL

scor

es

54

3.2.3 Immigrant status and CIL achievement

With increasing globalization and population mobility, many studies, including

ICILS  2013,  seek  to  find  out  the  extent  to  which  students’  immigrant  status  affects  their academic achievement.

Immigrant status is often associated with lower achievement for two key

reasons: the language of instruction and testing may not be the language used at

home; and the fact that recent immigrants may have a lower socioeconomic status.

In ICILS 2013, there are three sets of questions to elicit these three background

factors separately.

A student with immigrant background is defined in this Study as one for

whom both parents (if in a two-parent household) or the one parent (if in single-

parent household) were born in another country. As can be expected, the

percentage of students with an immigrant background differs greatly across

countries. In some countries, this figure could be close to zero, as in the case of

Korea. Hong Kong students reported the highest percentage with an immigrant

background, at 37%. Australia also has a relatively high percentage of students

(25%) reporting an immigrant background. Internationally, students with an

immigrant background scored on average 29 points lower than those without.

However, for Hong Kong and Australia, students with immigrant background

scored slightly higher than those without, but statistically non-significant.

3.2.4 Language use at home and CIL achievement

While 37% of the participating students in Hong Kong reported having an

immigrant background, only 11% reported that the test language was not the

language spoken at home. This is probably because in Hong Kong each student

could choose to do their performance assessment in either English or Chinese,

irrespective of the language medium of instruction used in their school. Most of

the students with immigrant background were from Mainland China. Most of the

non-ethnic Chinese students studying in publicly funded schools in Hong Kong

come from South Asian countries such as Nepal, Bangladesh, Pakistan and India.

Hong Kong students who did not speak the test language at home had a

significantly lower CIL score (26 points) compared with those who did. This

difference is similar to the international average, which is 31 points lower.

Australia has 11% of their students reporting an immigrant background, who also

scored lower than those without immigrant background.

However, the difference in score was only 8, and was not statistically

different. For Korea, only 1% of their students did not speak the test language at

home, and the sample size for this group was too small to give a reliable estimate

of the CIL score.

55

3.2.5 Socioeconomic background and CIL achievement

Socioeconomic status (SES) is an important construct often found in the education literature, but there is no commonly agreed method for how to construct measures of SES. In ICILS 2013, there were three sets of questions related to SES: highest level of   parental   education   reached,   parent’s   occupation   status   and   home   literacy resources  (i.e.  number  of  books  at  home).  Based  on  students’  responses   to  these  questions,   the   international   research   team   computed   an   index   of   students’  socioeconomic background (S_NISB). The exact computation for this index differs across countries, depending on the principal component analysis of the variables.

Tables 3.4 and 3.5 show the percentages of students at different parental education levels and with different levels of home literacy respectively, and the mean CIL score for each of the SES groups. For Hong Kong in comparison to Australia, Korea and the international mean.

As expected, in all the participating countries, students with higher SES were found to perform better than those with lower SES. However, similar to the case of the students’   educational   aspiration,   Hong   Kong   students   had   the   smallest  difference in CIL scores between the lowest and highest SES category, and this is true irrespective of which of the three SES indicators, parental education, parental occupation or home literacy is used.

3.2.6 Home ICT resources and CIL achievement

ICT resources at home can be very broad, and can include access to computers and the Internet as well as learning resources and support available at home. The present study has taken a simple approach and used only two indicators for this construct: the number of computers at home and access to the Internet at home. By computers, the survey included desktops, notebooks, netbooks and tablets. Internet access included dial-up connection, broadband and mobile phone network. There is wide variation across countries in terms of home ICT resources.

Table 3.6 presents the percentages of students with different numbers of computers at home and their respective CIL scores Internationally, 48% of students reported that they have three or more computers at home while 6% reported having none. Of the 21 systems participating in ICILS 2013, there are six systems where more than 80% of students have three or more computers at home, including Australia. For Hong Kong this figure is 55%, which is much higher than the corresponding percentage in Korea, which is only 32%. Table 3.3 further shows that in both Hong Kong and Korea, 2% of the students did not have any computer at home while the corresponding figure for Australia was only 1%. In all cases, on average, students with a higher number of computers at home had higher CIL scores, which probably is a result that confounds with the SES of the students

56

Hon

g K

ong

(SAR

)19

(1.4

)49

6(8

.7)

50(1

.3)

511

(6.4

)10

(1.0

)52

3(9

.2)

21(1

.5)

516

(11.

9)20

(9.3

)

ICIL

S 2

013

aver

age

15(0

.2)

453

(2.9

)33

(0.3

)49

0(1

.2)

17(0

.2)

504

(1.5

)35

(0.3

)52

5(1

.3)

72(3

.1)

Aust

ralia

11(0

.7)

506

(5.1

)22

(0.7

)51

8(3

.5)

22(0

.8)

539

(3.3

)46

(1.0

)56

4(2

.8)

58(5

.4)

Kor

ea, R

ep. o

f1

(0.3

)50

7(1

6.2)

31(1

.3)

525

(3.7

)9

(0.6

)51

9(6

.3)

59(1

.6)

545

(3.1

)39

(16.

0)*

Sta

tistic

ally

sig

nific

ant (

p<0.

05) c

oeffi

cien

ts in

bol

d.

Scor

e po

int d

iffer

ence

be

twee

n lo

wes

t and

hig

hest

ca

tego

ry

%

CIL

scor

e di

ffere

nce

(Uni

vers

ity

educ

atio

n -

low

er

seco

ndar

y or

be

low

)%

Mea

n CI

L sc

ore

%M

ean

CIL

scor

eM

ean

CIL

scor

e%

Mea

n CI

L sc

ore

High

est P

aren

t Edu

catio

n

Low

er-s

econ

dary

ed

ucat

ion

or lo

wer

Uppe

r-sec

onda

ry

educ

atio

nPo

st-s

econ

dary

non

-un

iver

sity

edu

catio

nTe

rtiar

y un

iver

ity

educ

atio

n

Educ

atio

nal s

yste

ms

() S

tand

ard

erro

rs a

ppea

r in

pare

nthe

ses.

Bec

ause

som

e re

sults

are

roun

ded

to th

e ne

ares

t who

le n

umbe

r, so

me

tota

ls m

ay a

ppea

r inc

onsi

sten

t.

025

5075

100

125

150

Diff

eren

ce s

tatis

tical

ly si

gnifi

cant

at .

05 le

vel.

Diff

eren

ce n

otst

atis

tical

ly si

gnifi

cant

.Stud

ents

in

high

est

cate

gory

scor

ehi

gher

tha

n in

lowe

st

Tabl

e 3. 4

The

per

cent

ages

of s

tude

nts a

t eac

h le

vel o

f par

enta

l edu

catio

n re

ache

d an

d th

eir r

espe

ctiv

e CIL

scor

es

Hon

g K

ong

(SAR

)14

(1.3

)47

4(9

.1)

23(1

.2)

504

(9.3

)33

(1.2

)51

9(5

.5)

31(1

.8)

519

(9.8

)44

(7.7

)

ICIL

S 2

013

aver

age

11(0

.2)

453

(1.9

)23

(0.3

)47

9(1

.3)

32(0

.3)

505

(1.0

)35

(0.3

)52

3(1

.4)

70(2

.1)

Aust

ralia

9(0

.5)

482

(5.3

)13

(0.7

)51

0(4

.5)

31(0

.9)

539

(3.2

)47

(1.1

)56

3(2

.2)

82(5

.5)

Kor

ea, R

ep. o

f7

(0.5

)47

0(6

.5)

6(0

.5)

512

(9.0

)21

(0.8

)53

1(3

.9)

66(1

.0)

547

(2.7

)76

(6.6

)

* Sta

tistic

ally

sign

ifica

nt (p

<0.0

5) c

oeffi

cien

ts in

bol

d.()

Sta

ndar

d er

rors

app

ear i

n pa

rent

hese

s. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

Scor

e po

int d

iffer

ence

be

twee

n lo

wes

t and

hig

hest

ca

tego

ry

%M

ean

CIL

scor

e%

Mea

n CI

L sc

ore

%M

ean

CIL

scor

e%

Mea

n CI

L sc

ore

Hom

e Li

tera

cy

Cate

gory

1

(0 to

10

book

s)Ca

tego

ry 2

(11

to 2

5 bo

oks)

CIL

scor

e di

ffere

nce

(hig

hest

- lo

wes

t hom

e lit

erac

y ca

tego

ry)

Educ

atio

nal s

yste

ms

Cate

gory

3(2

6 to

100

boo

ks)

Cate

gory

4(>

100

book

s)

025

5075

100125150

Diff

eren

ce s

tatis

tical

ly si

gnifi

cant

at .

05 le

vel.

Diff

eren

ce n

otst

atis

tical

ly si

gnifi

cant

.Stud

ents

in

high

est

cate

gory

scor

ehi

gher

tha

n in

lowe

st

Tabl

e 3.5

The

per

cent

ages

of s

tude

nts a

t eac

h le

vel o

f hom

e lite

racy

and

thei

r res

pect

ive C

IL sc

ores

57

Hon

g K

ong

(SAR

)2

(0.4

)42

2(2

0.6)

19(1

.1)

501

(8.2

)24

(1.0

)50

5(7

.9)

55(1

.6)

518

(8.1

)95

(20.

1)

ICIL

S 2

013

aver

age

6(0

.2)

420

(3.9

)21

(0.3

)48

5(2

.0)

24(0

.2)

502

(1.2

)48

(0.3

)51

7(1

.2)

94(4

.0)

Aust

ralia

1(0

.2)

440

(14.

8)4

(0.4

)48

7(9

.0)

10(0

.6)

523

(4.4

)85

(0.8

)54

8(2

.2)

108

(15.

0)

Rep

ublic

of K

orea

2(0

.2)

474

(16.

1)33

(1.0

)52

7(3

.7)

33(1

.0)

539

(3.8

)32

(1.0

)54

6(3

.4)

72(1

5.0)

* S

tatis

tical

ly s

igni

fican

t (p<

0.05

) coe

ffici

ents

in b

old.

() S

tand

ard

erro

rs a

ppea

r in

pare

nthe

ses.

Bec

ause

som

e re

sults

are

roun

ded

to th

e ne

ares

t who

le n

umbe

r, so

me

tota

ls m

ay a

ppea

r inc

onsi

sten

t.

Scor

e po

int d

iffer

ence

bet

wee

n lo

wes

t and

hig

hest

cat

egor

y

%M

ean

CIL

scor

e

CIL

scor

e di

ffere

nce

(hig

hest

- lo

wes

t hom

e lit

erac

y ca

tego

ry)

No c

ompu

ter

One

com

pute

rTw

o co

mpu

ters

Thre

e or

mor

e co

mpu

ters

%M

ean

CIL

scor

e%

Mea

n CI

L sc

ore

%M

ean

CIL

scor

eEd

ucat

iona

l sys

tem

s0

2550

75100125150

Stu

dent

s in

hi

ghes

t ca

tego

rysc

ore

high

er th

an in

lo

wes

t

Diff

eren

ce s

tatis

tical

ly si

gnifi

cant

at .

05 le

vel.

Diff

eren

ce n

otst

atis

tical

ly si

gnifi

cant

.

Tabl

e 3.6

The

per

cent

ages

of s

tude

nts w

ith d

iffer

ent n

umbe

rs o

f com

pute

rs a

t hom

e and

thei

r res

pect

ive C

IL sc

ores

58

Figures in Table 3.6 also show that the biggest difference is between students

who have no computers at home and those who have access to at least one

computer. For both Hong Kong and Korea, the increases in mean CIL scores with

increasing numbers of computers were relatively small.

There were only 1% of students in Hong Kong who reported not having

Internet access at home, which is lower than the 2% who reported not having a

computer at home. This indicates that for some Hong Kong students, their access

to the Internet at home was only via their mobile phones. As the actual number for

this group of students was too small, it was not possible to obtain a reliable estimate

for their CIL. In addition to whether there is access, the means by which students

gain access to the Internet at home may also have influence on their CIL

achievement. However, responses from students indicate that many of them were

not clear about their exact means of access at home. Hence, any finer level of

analysis is not pursued.

3.3 Influence of  Students’  ICT  Self-efficacy and Interest

In  addition  to  students’  personal  and  family  background,  their  own self-perceived

competence with respect to ICT skills and interest in ICT use would likely have

impact on their CIL achievement. Questions with multiple items requiring

responses on a Likert scale were used to generate scale scores for each of these

perception related constructs.

3.3.1 Self-efficacy in basic ICT skills and CIL achievement

In the survey, students were asked to indicate how well they thought they could

handle each of 13 computer-based tasks on a Likert scale with three response

categories:  “I  know  how  to  do  this”,  “I  could  work  out  how  to  do  this”,  and  “I  do  not  think  I  could  do  this”.  Two  scales  can  be  constructed  from  these  13  tasks,  one  on Self-efficacy in Basic ICT Skills (S_BASEFF) from responses to six basic level tasks,

and the other on Self-efficacy in Advanced ICT Skills (S_ADVEFF) from responses to

seven relatively more advanced tasks. The six tasks comprising the S_BASEFF scale

are:

x Search for and find a file on your computer;

x Edit digital photographs or other graphic images;

x Create or edit documents (for example assignments for school);

x Search for and find information you need on the Internet;

x Create a multimedia presentation (with sound, pictures, or video); and

x Upload text, images or video to an online profile.

59

79(1

.4)

65

(1.3

)

75(1

.5)

79

(1.5

)

66(1

.5)

72

(1.5

)48

(0.5

)0.

40(0

.03)

87(0

.2)

73(0

.3)

81(0

.2)

89(0

.2)

64(0

.3)

77(0

.3)

50(0

.1)

0.32

(0.0

1)

91(0

.6)U

69(0

.7)V

90(0

.6)U

94(0

.5)U

73(0

.8)U

83(0

.7)U

52(0

.2)×

0.36

(0.0

2)

87(0

.7)

61(1

.1)T

80(0

.8)

87(0

.7)V

52(1

.1)T

73(1

.0)V

49(0

.2)Ø

0.42

(0.0

2)*

Perc

enta

ges

refle

ct s

tude

nts

that

sel

ecte

d "c

onfid

ent"

* St

atis

tical

ly s

igni

fican

t (p<

0.05

) coe

ffic

ient

s in

bol

d.( )

St

anda

rd e

rror

s ap

pear

in p

aren

thes

es. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent. #

Mea

n of

sca

le s

core

=50

SM

ore

than

10

perc

enta

ge p

oint

s ab

ove

ICIL

S av

erag

eÏ

Mor

e th

an 3

sco

re p

oint

s ab

ove

ICIL

S av

erag

eU

Sign

ifica

ntly

abo

ve IC

ILS

aver

age

×Si

gnifi

cant

ly a

bove

ICIL

S av

erag

eV

Sign

ifica

ntly

bel

ow IC

ILS

aver

age

ØSi

gnifi

cant

ly b

elow

ICIL

S av

erag

eT

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

belo

w IC

ILS

aver

age

ÐM

ore

than

3 s

core

poi

nts

belo

w IC

ILS

aver

age

Ove

rall

stud

ents

' av

erag

e sc

ale

scor

e fo

r sel

f-ef

ficac

y in

ba

sic

ICT

skill

s (S

_BA

SEFF

) #

Corr

elat

ion

coef

ficie

nts

for

CIL

with

S_

BASE

FF

Aust

ralia

Kor

ea, R

ep. o

f

Stud

ents

' sel

f-effi

cacy

in b

asic

ICT

skill

s th

at a

re in

clud

ed in

sca

le s

core

S_B

ASE

FF

Crea

te a

mul

ti-m

edia

pr

esen

tatio

n (w

ith s

ound

, pi

ctur

es, o

r vi

deo)

Sear

ch fo

r and

fin

d a

file

on

your

com

pute

r

Edit

digi

tal

phot

ogra

phs

or

othe

r gra

phic

im

ages

Crea

te o

r edi

t do

cum

ents

(for

ex

ampl

e as

sign

men

ts fo

r sc

hool

)

Sear

ch fo

r and

fin

d in

form

atio

n yo

u ne

ed o

n th

e In

tern

et

Uplo

ad te

xt,

imag

es o

r vi

deo

to a

n on

line

prof

ileEd

ucat

iona

l Sys

tem

Hon

g K

ong

(SAR

)

ICIL

S 2

013

aver

age

Tabl

e 3.7

The

per

cent

ages

of s

tude

nts c

onfid

ent i

n pe

rfor

min

g ea

ch b

asic

ICT

skill

s tas

k, S

_BA

SEFF

and

cor

rela

tion

with

CIL

scor

es

60

To   compute   students’   self-confidence in each of the above tasks, students

who  responded  “I  could  work  out  how  to  do  this”  and  “I  do  not  think  I  could  do  this”  were   collapsed   into   one   category   as   not   confident.   Table   3.7   presents   the  percentages of students who reported to be confident in each of these six tasks. It

can be seen that internationally and in Hong Kong, students had the highest

confidence in file searching on their own computers and information search on the

Internet. There is somewhat more variations in terms of which tasks students were

least confident in. Overall, students found editing digital photographs or other

graphic images and creating multimedia presentations to be the most difficult. In

Hong Kong, each of the six task areas had at least 65% of students claiming

confidence in their ability to perform them. Further, this self-proclaimed

confidence was relatively uniform across tasks for Hong Kong students.

In terms of the overall self-efficacy score for basic ICT skills (S_BASEFF,

coefficient alpha=0.76), Hong Kong is very similar to the international average, as

are both Australia and Korea. Furthermore this score significantly correlates with

the  students’  CIL  scores  for  all  participating  countries.  The  correlation  coefficient  is about 0.4 for all three systems, as presented in Table 3.7.

3.3.2 Self-efficacy in advanced ICT skills and CIL achievement

Seven tasks were included in the construction of the Self-efficacy in Advanced ICT Skills scale (S_ADVEFF):

x Use software to find and get rid of viruses;

x Create a database (for example using [Microsoft Access ®]);

x Build or edit a webpage;

x Change the settings on your computer to improve the way it operates or to

fix problems;

x Use a spreadsheet to do calculations, store data or plot a graph;

x Create a computer program or macro (for example in [Basic]); and

x Set up a computer network.

Table 3.8 presents the percentages of students who reported to be confident

in each of the seven advanced ICT tasks. It is clear that these percentages are much

lower compared to those in Table 3.7 for the basic ICT tasks. The task that students

were least confident about is to create a computer program or macro. Interestingly,

students in two of the highest ICILS performing countries, Australia and Korea,

reported a lower level of confidence in this task compared to the international

average. Hong Kong students report the second highest confidence in this task

(28%), which is below that of Turkey and the same as Slovenia. It is thus apparent

that countries with high CIL performance did not put emphasis on the teaching of

programming.

61

There are two advanced ICT tasks for which more than half of the students

internationally have confidence in handling: changing the settings on a computer

to fix problems or improve its performance, and using a spreadsheet for

computation or graphing. Interestingly, less than 40% of Korean students reported

having confidence in either of these two tasks, indicating that these two skills may

be curriculum related. Generally, Hong Kong students report a higher level of

confidence than the international average in all of the advanced ICT tasks, except

for setting up a computer network. It is noteworthy that Korea is the only country

for which more than half of the students (56%) reported having confidence in

setting up a computer network. An index for self-efficacy in advanced ICT skills

was similarly computed (S_ADVEFF, coefficient alpha=0.80) with an international

mean of 50 and one standard deviation of 10 points. Results in Table 3.8 show that

the national means for all the participating countries were very close to 50, which

is similar to the case with S_BASEFF. However, the picture becomes very different

when  we  examine  the  correlation  coefficients  between  students’  S_ADVEFF  and  CIL scores, which were much lower than those for S_BASEFF and were smaller

than 0.1 for most countries. The correlation coefficient was significant for only half

of the countries, with eight of these being negative out of the 21 participating

countries. As the advanced ICT tasks are more specialized, students are most likely

to have learnt them through specialized instruction, especially through computer-

related   subjects.   The   low   correlation   of   S_ADVEFF   with   students’   CIL   scores  indicates that (1) teaching about how to use computers is not adequate in fostering

students’  CIL  abilities,  and  (2)  it  is  not  necessary to possess advanced ICT skills to

be competent in CIL.

3.3.3 Interest and enjoyment in using ICT and CIL achievement

Data   on   students’   interest   and   enjoyment   in   using   ICT   was   gathered   through  responses to seven statements on a four-point Likert scale (response categories:

“strongly  agree,”  “agree,”  “disagree,”  “strongly  disagree”).  The  seven  statements  are as follows:

x It is very important to me to work with a computer;

x I think using a computer is fun;

x It is more fun to do my work using a computer than without a computer;

x I use a computer because I am very interested in the technology;

x I like learning how to do new things using a computer;

x I often look for new ways to do things using a computer; and

x I enjoy using the Internet to find out information.

Table  3.9  presents  the  percentages  of  students  who  indicated  “agree”  or  “strongly  agree”  with  each  of  the  above  statements  and  the  scale  scores  for  the   interest and enjoyment in computing scale (S_INTRST, coefficient alpha ≥  0.74  for  all  countries).  

62

Tabl

e 3.8

The

per

cent

ages

of s

tude

nts c

onfid

ent i

n pe

rfor

min

g ea

ch b

asic

ICT

skill

s tas

k, S

_AD

VEF

F an

d co

rrel

atio

n w

ith C

IL sc

ores

53(1

.2)

37

(1.4

)

42(1

.2)

59

(1.4

)

64(1

.5)

28

(1.3

)

29(1

.3)

51

(0.3

)

0.09

(0.0

)

47(0

.3)

30(0

.3)

38(0

.3)

57(0

.3)

54(0

.3)

21(0

.3)

35(0

.3)

50(0

.1)

0.04

(0.0

)

32(0

.9)T

24(0

.9)V

31(1

.0)V

59(0

.8)

50(1

.1)V

17(0

.7)V

27(0

.8)V

48(0

.2)Ø

0.04

(0.0

)

55(1

.0)U

25(0

.9)V

37(0

.8)

38(1

.0)T

35(1

.0)T

16(0

.8)V

56(1

.0)S

52(0

.2)×

0.13

(0.0

)*

Perc

enta

ges

refle

ct s

tude

nts

that

sel

ecte

d "c

onfid

ent"

* St

atis

tical

ly s

igni

fican

t (p<

0.05

) coe

ffic

ient

s in

bol

d.( )

St

anda

rd e

rror

s ap

pear

in p

aren

thes

es. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

#M

ean

of s

cale

sco

re =

50S

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

abov

e IC

ILS

aver

age

ÏM

ore

than

3 s

core

poi

nts

abov

e IC

ILS

aver

age

USi

gnifi

cant

ly a

bove

ICIL

S av

erag

e×

Sign

ifica

ntly

abo

ve IC

ILS

aver

age

VSi

gnifi

cant

ly b

elow

ICIL

S av

erag

eØ

Sign

ifica

ntly

bel

ow IC

ILS

aver

age

TM

ore

than

10

perc

enta

ge p

oint

s be

low

ICIL

S av

erag

eÐ

Mor

e th

an 3

sco

re p

oint

s be

low

ICIL

S av

erag

e

Ove

rall

stud

ents

' av

erag

e sc

ale

scor

e fo

r sel

f-ef

ficac

y in

ad

vanc

ed IC

T sk

ills

(S_A

DVEF

F) #

Corr

elat

ion

coef

ficie

nts

for

CIL

with

S_

ADV

EFF

Kor

ea, R

ep. o

f

Use

a sp

read

shee

t to

do c

alcu

latio

ns,

stor

e da

ta o

r plo

t a

grap

h

Crea

te a

co

mpu

ter

prog

ram

or

mac

ro (f

or

exam

ple

in

[Bas

ic, V

isua

l Ba

sic]

)

Use

softw

are

to

find

and

get r

id o

f vi

ruse

s

Crea

te a

da

taba

se (f

or

exam

ple

usin

g [M

icro

soft

Acce

ss ®

])Bu

ild o

r edi

t a

web

page

Chan

ge th

e se

tting

s on

you

r co

mpu

ter t

o im

prov

e th

e w

ay

it op

erat

es o

r to

fix p

robl

ems

Stud

ents

' sel

f-effi

cacy

in a

dvan

ced

ICT

skill

s th

at a

re in

clud

ed in

sca

le s

core

S_A

DVEF

F

Educ

atio

nal S

yste

m

Hon

g K

ong

(SAR

)

ICIL

S 2

013

aver

age

Aust

ralia

Set u

p a

com

pute

r ne

twor

k

63

Tabl

e 3.9

Per

cent

ages

of s

tude

nts a

gree

ing

with

stat

emen

ts a

bout

com

pute

rs, S

_IN

TRST

and

cor

rela

tion

with

CIL

scor

es

92(0

.7)

92

(1.0

)

77(1

.0)

69

(1.3

)

86(1

.1)

80

(1.1

)

93(0

.8)

50(0

.4)

0.12

(0.1

)

89(0

.2)

91(0

.2)

83(0

.2)

63(0

.3)

91(0

.2)

78(0

.2)

92(0

.2)

50(0

.1)

0.08

(0.0

)

88(0

.6)

93(0

.5)U

85(0

.6)U

65(0

.9)U

91(0

.5)

75(0

.8)V

93(0

.5)

49(0

.2)Ø

0.11

(0.0

)

79(0

.9)T

88(0

.6)V

76(0

.9)V

42(1

.3)T

86(0

.8)V

67(0

.9)T

88(0

.7)V

46(0

.3)Ð

0.11

(0.0

)*

Perc

enta

ges

refle

ct s

tude

nts

that

sel

ecte

d "S

trong

ly A

gree

" or "

Agr

ee".

* St

atis

tical

ly s

igni

fican

t (p<

0.05

) coe

ffic

ient

s in

bol

d.( )

St

anda

rd e

rror

s ap

pear

in p

aren

thes

es. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

#M

ean

of s

cale

sco

re =

50S

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

abov

e IC

ILS

aver

age

ÏM

ore

than

3 s

core

poi

nts

abov

e IC

ILS

aver

age

USi

gnifi

cant

ly a

bove

ICIL

S av

erag

e×

Sign

ifica

ntly

abo

ve IC

ILS

aver

age

VSi

gnifi

cant

ly b

elow

ICIL

S av

erag

eØ

Sign

ifica

ntly

bel

ow IC

ILS

aver

age

TM

ore

than

10

perc

enta

ge p

oint

s be

low

ICIL

S av

erag

eÐ

Mor

e th

an 3

sco

re p

oint

s be

low

ICIL

S av

erag

e

Aust

ralia

Kor

ea, R

ep. o

f

I ofte

n lo

ok

for n

ew w

ays

to d

o th

ings

us

ing

a co

mpu

ter

It is

ver

y im

port

ant t

o m

e to

wor

k w

ith a

co

mpu

ter

I thi

nk u

sing

a

com

pute

r is

fun

It is

mor

e fu

n to

do

my

wor

k us

ing

a co

mpu

ter

than

with

out a

co

mpu

ter

I use

a

com

pute

r be

caus

e I a

m

very

in

tere

sted

in

the

tech

nolo

gy

I lik

e le

arni

ng

how

to d

o ne

w th

ings

us

ing

a co

mpu

ter

Corr

elat

ion

coef

ficie

nts

for

CIL

with

S_

INTR

ST

Stud

ents

' int

eres

t and

enj

oym

ent i

n us

ing

com

pute

rs th

at a

re in

clud

ed in

sca

le s

core

S_I

NTRS

T

Educ

atio

nal S

yste

m

Hon

g K

ong

(SAR

)

ICIL

S 2

013

aver

age

I enj

oy u

sing

th

e In

tern

et

to fi

nd o

ut

info

rmat

ion

Ove

rall

stud

ents

' av

erag

e sc

ale

scor

e fo

r in

tere

st a

nd

enjo

ymen

t in

usin

g co

mpu

ters

(S

_INT

RST)

#

64

It  is  interesting  to  note  that  the  statement  “I use a computer because I am very interested in the technology”  had  the  lowest  percentage  agreement  among  all  seven  statements from students in all participating countries. Furthermore, international results presented in Fraillon et al. (2014, p.162) show that countries scoring the highest agreement for this statement (reaching 79% and above) were also countries that had the highest percentages of students not having access to any computer at home: Thailand (28%), Chile (7%) and Turkey (31%). This indicates that interest in the technology per se wears off for some students once they have ready access to computers. Over 90% of the students in Hong Kong find it important and fun to use a computer and enjoy finding information on the Internet.

Results  in  Table  3.9  also  show  that  Hong  Kong  students’  overall  interest  and enjoyment in computing scale score is similar to the international average. It is also observed  that  students’  S_INTRST scores were positively correlated with their CIL scores for most countries. For Hong Kong, Australia and Korea, the correlation coefficients were statistically significant at around 0.11 to 0.12.

3.4   Influence   of   Students’   ICT   Use   Experience   at   Home   and   in  School

Other  than  eliciting  students’  perceptions  of  their  own  self-efficacy in ICT skills, and interest and enjoyment in computing, the  survey  also  asked  about  students’  use of computers within and outside of school. In this section, we will report on how  Hong  Kong  students’  computer  use  compares  with  Australia,  Korea  and  the  international means, and if computer use correlates with students’   CIL  achievement.

3.4.1 Computer experience and CIL achievement

Students’  history  of  using  computers  affects  their  familiarity  with  ICT  and  hence  also their CIL proficiency. Table 3.10 presents the number of years of using computers as reported by the students. It shows clearly that students have generally used computers for a very substantial period of time. Internationally, 64% of students reported having used computers for at least five years. This figure is even higher for Hong Kong, at 72%, which is lower than Australia (78%) but slightly higher than Korea (69%). The percentages of students reporting less than one year of computing experience is at a low, single digit, except for Thailand (23%) and Turkey (22%). The average length of time these millennials, aged about 14, had been using computers was six, clearly matching the description of being the digital generation from this respect.

65

2(0

.4)

7(0

.7)

19(0

.9)

27(1

.0)

45(1

.4)

6(0

.1)

6(1

.2)

2(0

.8)

5(0

.2)

9(0

.2)

20(0

.2)

29(0

.3)

36(0

.3)

6(0

.0)

9(0

.3)

6(0

.3)

1(0

.2)

5(0

.4)

15(0

.6)

28(0

.8)

50(1

.1)

6(0

.0)

10(0

.7)

6(0

.9)

5(0

.4)

11(0

.7)

15(0

.8)

25(0

.9)

44(1

.1)

6(0

.1)

10(0

.7)

7(1

.0)

* St

atis

tical

ly s

igni

fican

t (p<

0.05

) coe

ffic

ient

s in

bol

d.( )

St

anda

rd e

rror

s ap

pear

in p

aren

thes

es. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

Leng

th o

f tim

e us

ing

com

pute

rsAv

erag

e le

ngth

of

time

usin

g co

mpu

ters

(y

ears

)

Effe

ct o

f com

pute

r ex

perie

nce

on

CIL

scor

e

Less

than

on

e ye

ar

At le

ast o

ne

year

but

le

ss th

an

thre

e ye

ars

At le

ast

thre

e ye

ars

but l

ess

than

five

At le

ast f

ive

year

s bu

t le

ss th

an

seve

n ye

ars

Seve

n or

m

ore

year

s

Diffe

renc

e in

sco

re

poin

ts p

er

year

of

Varia

nce

expl

aine

dEd

ucat

iona

l Sys

tem

Hon

g K

ong

(SAR

)

ICIL

S 2

013

aver

age

Aust

ralia

Kor

ea, R

ep. o

f

Tabl

e 3.1

0 Pe

rcen

tage

s of s

tude

nts w

ith d

iffer

ent y

ears

of e

xper

ienc

e with

com

pute

rs

66

The last two columns in Table 3.10 show the bivariate regression results

between   students’   years   of   computer   experience   and   their   CIL   scores. In all

countries, there is a positive association between years of computer experience and

CIL score, which is statistically significant except for Germany. There is an average

increase of nine score points associated with one additional year of computer

across all countries, with years of computer experience accounting for 6% of the

total CIL score variance. For Hong Kong, the association is lower, with an increase

of six score points per additional year of computer experience.

In  addition  to  finding  out  about  students’  years  of  computer  experience,  the  survey asked students about the frequency with which they use computers at

various locations. Defining computer use at least once a week as frequent use,

Table 3.11 records the percentages of students who reported frequent computer

use at home, at school and at other places (such as local library, Internet café).

Across all countries, a majority of students used computers at home at least

once a week (average=87%), whereas only 54% did so at school, and only 13% used

computers frequently at other places.

Australia had the highest percentage of students with frequent computer use

at school, at 81%. Korea and Thailand had the highest percentage of students

reporting frequent use of computers outside of the home or school, at 30% and 31%

respectively.

Korea stands out as the only country where more students reported frequent

use of computers at places other than school. This could be related to the high levels

of  participation  of  school  children  in  private  tutoring  and  the  Korean  government’s  efforts to reduce the financial burden of private tutoring by launching in 2004 the

Cyber Home Learning System (CHLS) to reduce the need for fee-paying tutoring

(Bray and Lykins, 2012). In 2013, an average of 60.2% of Korean students from

Elementary to High School and 50.5% middle school students participated in after-

school programs (KOSIS, 2014).

For Hong Kong, the percentages of students reporting frequent use of

computers at various locations were very similar to the international average, and

the percentage of students using computers frequently in places other than at home

or school were lower than the international average.

67

Table 3.11 Percentages of students with frequent computer use (i.e. at least once a week) at home, school and other places

3.4.2 Use of computers for and at school

While it is expected that the number of years of computer use experience and the frequency   of   use   would   influence   students’   CIL   proficiency,   the   nature   and  purposes of use probably also affect their CIL outcomes. In this section, we report on   the  context,  purpose  and   tasks   for  students’  use  of  computers  at  school,  and  how these factors influence their CIL scores. Use of computers in different subject areas (S_USELRN): The survey asked students how often they used computers during lessons in eight subject  areas,  giving  them  five  response  options:  “never,”  “in  some  lessons,”  “in  most   lessons,”   “in   every   or   almost   every   lesson,”   and   “I   don’t   study   this  subject/these   subjects.”   Table   3.12   presents   the   percentages   of   students   who  responded  that  they  used  computers  “in  most  lessons”  or  “in  every  or  almost  every  lesson.”  Those  responding,  “I  don’t  study  this  subject/these  subjects,”  are  treated  as missing responses.

The results of the survey show that internationally, for nearly all school subject areas, the percentages of students who reported that computers were used in most or almost every lesson were relatively low, between 11% and 21%. The only exception was computer related subjects such as information technology, registering 56%.

88 (1.0) 57 (2.0) 8 (0.7)

87 (0.2) 54 (0.5) 13 (0.2)

87 (0.7) 81 (1.3) S 9 (0.5) V

71 (1.2) T 18 (2.1) T 30 (1.3) S

( ) Standard errors appear in parentheses. Because some results are rounded to the nearest w hole number, some totals may appear inconsistent.

S More than 10 percentage points above ICILS averageU Signif icantly above ICILS averageV Signif icantly below ICILS averageT More than 10 percentage points below ICILS average

At home At school

Percentage of students using a computer at least once a week

At other places (for example local library,

internet cafe)

Educational System

Hong Kong (SAR)

ICILS 2013 average

Australia

Korea, Rep. of

68

12(1

.1)

13

(1.2

)

9(1

.2)

15

(1.1

)

15(1

.4)

45

(0.5

)-0

.13

(0.0

)11

(1.1

) 81

(1.6

)

8(1

.0)

16(0

.3)

17(0

.3)

14(0

.3)

21(0

.3)

20(0

.3)

50(0

.1)

0.00

(0.0

)11

(0.2

)56

(0.4

)11

(0.2

)

34(1

.8)S

24(1

.9)U

23(1

.8)U

34(1

.8)S

42(1

.7)S

57(0

.3)Ï

0.15

(0.0

)14

(0.9

)U

58(1

.8)

14(0

.8)

U

25(1

.2)U

37(1

.2)S

15(1

.0)

30(1

.2)U

22(1

.2)

49(0

.3)Ø

0.13

(0.0

)18

(0.8

)U

33(1

.7)T

19(0

.9)

U

*Pe

rcen

tage

s re

flect

stu

dent

s th

at s

elec

ted

"in m

ost l

esso

ns" o

r "in

eve

ry o

r alm

ost e

ery

less

on"

*St

atis

tical

ly s

igni

fican

t (p<

0.05

) coe

ffic

ient

s in

bol

d.( )

St

anda

rd e

rror

s ap

pear

in p

aren

thes

es. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

#M

ean

of s

cale

sco

re =

50S

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

abov

e IC

ILS

aver

age

ÏM

ore

than

3 s

core

poi

nts

abov

e IC

ILS

aver

age

USi

gnifi

cant

ly a

bove

ICIL

S av

erag

e×

Sign

ifica

ntly

abo

ve IC

ILS

aver

age

VSi

gnifi

cant

ly b

elow

ICIL

S av

erag

eØ

Sign

ifica

ntly

bel

ow IC

ILS

aver

age

TM

ore

than

10

perc

enta

ge p

oint

s be

low

ICIL

S av

erag

eÐ

Mor

e th

an 3

sco

re p

oint

s be

low

ICIL

S av

erag

e

Hong

Kon

g (S

AR)

ICIL

S 2

013

aver

age

Aust

ralia

Kor

ea, R

ep. o

f

Stud

ents

' use

of I

CT d

urin

g le

sson

s at

sch

ool t

hat a

re in

clue

d in

sca

le s

core

S_

USEL

RN

Lang

uage

arts

: te

st la

ngua

ge

Lang

uage

arts

: fo

reig

n or

oth

er

natio

nal

lang

uage

sM

athe

mat

ics

Scie

nces

(g

ener

al

scie

nce

and/

or

phys

ics,

ch

emis

try,

biol

ogy,

ge

olog

y, e

arth

sc

ienc

es)

Educ

atio

nal S

yste

m

Crea

tive

arts

(v

isua

l arts

, m

usic

, dan

ce,

dram

a et

c.)

Info

rmat

ion

tech

nolo

gy,

com

pute

r st

udie

s or

si

mila

r

Oth

er (p

ract

ical

or

voca

tiona

l sub

ject

s,

mor

al/e

thic

s, p

hysi

cal

educ

atio

n, h

ome

econ

omic

s, p

erso

nal

and

soci

al

deve

lopm

ent)

Hum

an

scie

nces

/H

uman

ities

(h

isto

ry,

geog

raph

y,

civi

cs, l

aw,

econ

omic

s et

c.)

Ove

rall

stud

ents

' av

erag

e sc

ale

scor

e fo

r use

of

ICT

durin

g le

sson

s at

sc

hool

(S

_USE

LRN)

#

Corr

elat

ion

coef

ficie

nts

for

CIL

with

S_

USEL

RN

Tabl

e 3.1

2 Pe

rcen

tage

s of s

tude

nts u

sing

com

pute

rs in

mos

t les

sons

or a

lmos

t eve

ry le

sson

in d

iffer

ent l

earn

ing

area

s, S

_UEL

RN

and

as

soci

atio

n w

ith C

IL sc

ore

69

For Hong Kong, even though schools are relatively well catered for in terms of computer: student ratio (see chapter 4 for details), the percentages of students reporting frequent use of computers in the teaching of school subject areas most studied across all participating countries (these are languages (test language and an additional language), mathematics, sciences and humanities, to be referred to as the key school subjects) were all substantially lower than the international average.

The highest percentages for Hong Kong were in the humanities and sciences, both at 15%. The lowest use was in Mathematics, at only 9%. This contrasts starkly with the situation in Australia and Korea, which are among the top performing countries in ICILS. Australia reported significantly higher percentages than the international average of reported use in most or almost every lesson for all five key school  subjects,  and  Korea’s  corresponding  percentages  were  also  higher  than  the  international average, with three being statistically significant.

A composite scale indicator for frequency of ICT use during lessons in the five key school subjects (S_USELRN) was computed using Rasch partial credit model   (international   mean=50,   standard   deviation=10,   Cronbach’s   alpha  reliability=0.81). The results shown in Table 3.12 are worrying not only because the S_USELRN score for Hong Kong was very low. More seriously, the correlation coefficient between S_USELRN and CIL score was negative at -0.13 for Hong Kong, whereas the correlations for Australia and Korea were both positive, at 0.15 and 0.13 respectively.

This finding warrants further investigation. Why is it that students reporting more use of ICT in lessons of key school subjects had lower CIL achievement? Since there is a large between-school   variance   in   students’   academic   achievement   at  entrance into secondary school, this negative correlation could indicate that schools with lower student achievement at intake were more willing to use ICT in teaching,  or  that  the  way  ICT  is  used  in  lessons  had  a  negative  impact  on  students’  CIL development. It could also be that both conditions are at work to bring about this negative result. Use of computers for study purposes (S_USESTD) The survey also asked students about how often they use computers for eight different study related purposes, using four  response  options:  “never,”  “less  than  once  a  month,”  “at  least  once  a  month  but  not  every  week,”  and  “at  least  once  a  week.”  These  purposes  are  not  subject  specific,  and  can  be  grouped  under  three  categories:

1. For purposes requiring some student self-direction, comprising: x Preparing reports or essays; x Preparing presentations;

70

2. Organizing your time and work; and x Writing about your learning. x For collaboration, comprising: x Working with other students from your own school; and x Working with other students from other schools.

3. For drill and practice type of purpose, comprising: x Completing worksheets or exercises; and x Completing tests.

Table 3.13 presents the percentages of students who report using computers

at least once a month for each of the above purposes. The Table also presents the composite scale score for the extent of using computers for school-related purposes (S_USESTD, computed using Rasch partial credit model with international average=50,   standard  deviation=10,   and  Cronbach’s   alpha   scale   reliability=0.83), and  the  correlation  of  the  scale  scores  with  students’  CIL  scores.

Internationally, the highest reported use is for preparation of reports or essays, and Australia reported the highest percentage for this purpose, at 70%. On the other hand, Hong Kong students reported the highest percentage use for completing worksheets or exercises (51%), which is also the only purpose for which more than half of the students reported using computers at least once a month.

A more careful examination shows that  Hong  Kong  students’  computer  use  frequency for all four study purposes that require some student self-direction were lower than the international average, while use for collaboration with students from   within   or   outside   the   students’   own   school   were   both higher than the international mean. It is interesting to note that the percentage of computer use for study purposes reported by Korean students were all significantly lower than the international average, while the opposite was true for Australian students except for collaborating with students from other schools.

Despite the wide variations in frequency of computer use for study purposes across Hong Kong, Australia and Korea, all three systems record a similarly fixed and positive correlation between S_USESTD, which is much higher than that for the corresponding international correlation.

This  indicates  that  at  least  for  these  three  systems,  students’  use  of  computers  for study purposes has a positive association with CIL scores, irrespective of the exact level of average use. Furthermore, it may not be meaningful to consider the international correlation coefficient due to the wide differences in the actual levels of use across different countries.

71

43(1

.7)

36

(1.4

)

25(1

.4)

17

(1.1

)

44(1

.8)

19

(1.2

)

51(1

.3)

27

(1.3

)50

(0.4

)0.

11(0

.0)

45(0

.3)

44(0

.4)

30(0

.3)

19(0

.2)

40(0

.3)

13(0

.2)

39(0

.3)

33(0

.3)

50(0

.1)

0.04

(0.0

)

70(1

.0)S

68(1

.1)S

45(1

.2)S

22(0

.9)U

56(1

.2)S

11(0

.6)V

64(1

.3)S

44(1

.1)S

54(0

.3)Ï

0.15

(0.0

)

21(1

.0)T

23(1

.1)T

17(0

.8)T

16(0

.7)V

16(0

.8)T

11(0

.7)V

20(0

.8)T

17(0

.8)T

44(0

.3)Ð

0.13

(0.0

)*

Perc

enta

ges

refle

ct s

tude

nts

that

sel

ecte

d "A

t lea

st o

nce

a m

onth

but

not

eve

ry w

eek"

& "A

t lea

st o

nce

a w

eek"

*St

atis

tical

ly s

igni

fican

t (p<

0.05

) coe

ffic

ient

s in

bol

d.( )

St

anda

rd e

rror

s ap

pear

in p

aren

thes

es. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

#M

ean

of s

cale

sco

re =

50S

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

abov

e IC

ILS

aver

age

ÏM

ore

than

3 s

core

poi

nts

abov

e IC

ILS

aver

age

USi

gnifi

cant

ly a

bove

ICIL

S av

erag

e×

Sign

ifica

ntly

abo

ve IC

ILS

aver

age

VSi

gnifi

cant

ly b

elow

ICIL

S av

erag

eØ

Sign

ifica

ntly

bel

ow IC

ILS

aver

age

TM

ore

than

10

perc

enta

ge p

oint

s be

low

ICIL

S av

erag

eÐ

Mor

e th

an 3

sco

re p

oint

s be

low

ICIL

S av

erag

e

Kor

ea, R

ep. o

f

Ove

rall

stud

ents

' av

erag

e sc

ale

scor

e fo

r use

of

ICT

for s

tudy

pu

rpos

es

(S_U

SEST

D) #

Corr

elat

ion

coef

ficie

nts

for

CIL

with

S_

USES

TD

Educ

atio

nal S

yste

m

Hon

g K

ong

(SAR

)

ICIL

S 2

013

aver

age

Aust

ralia

Wor

king

with

ot

her s

tude

nts

from

oth

er

scho

ols

Com

plet

ing

wor

kshe

ets

or

exer

cise

sCo

mpl

etin

g te

sts

Stud

ents

' use

of I

CT fo

r stu

dy p

urpo

ses

that

are

incl

uded

in s

cale

sco

re S

_USE

STD

Prep

arin

g re

ports

or

essa

ys

Prep

arin

g pr

esen

tatio

ns

Org

anis

ing

your

tim

e an

d w

ork

Writ

ing

abou

t yo

ur le

arni

ng

Wor

king

with

ot

her s

tude

nts

from

you

r ow

n sc

hool

Tabl

e 3.1

3 Pe

rcen

tage

s of s

tude

nts u

sing

com

pute

rs fo

r stu

dy p

urpo

ses a

t lea

st o

nce a

mon

th, S

_USE

STD

and

ass

ocia

tion

with

CIL

scor

e

72

Learning of CIL related tasks at school (S_TSKLRN) There is one question in the survey that asked students whether or not they have learnt each of the following eight CIL-related tasks at school, which could be categorized according to the two CIL competency strands:

1. Collecting and managing information:

x Accessing information with a computer; x Looking for different types of digital information on a topic; x Deciding where to look for information about an unfamiliar topic; x Working out whether to trust information from the Internet; x Deciding what information is relevant to include in school work; x Organizing information obtained from Internet sources.

2. Producing and exchanging information x Presenting information for a given audience or purpose with a computer; x Providing references to Internet sources.

Table 3.14 presents the percentages of students reporting having learnt each

of these tasks at school. The results show that in Hong Kong and internationally, accessing information with a computer was the one task that more than 80% of students have learnt in school. This was also the task with the highest percentage of students reporting having learnt at school, although that was only 74%. The task that Hong Kong students were least prepared for was on evaluating information—working out whether to trust information from the Internet, with only 53% reporting having an opportunity to learn this at school. Australian students reported significantly higher percentages of having learnt in school all of the eight CIL- related tasks compared to the international means, whereas all the corresponding percentages for Korean students were significantly lower than the international means. This observation is similar to the statistics on the percentages of students using computers for study purposes at least once a month reported earlier.

Rasch partial credit model was also applied to compute a scale on students having learnt CIL-related tasks at school (S_TSKLRN, international average=50, standard  deviation=10,  and  Cronbach’s  alpha  scale  reliability=0.81).  Hong  Kong’s  score is below the international mean at 48, and Korea is even lower, at 46, while Australian is significantly higher, at 54. The correlation coefficients between S_TSKLRN and CIL scores were positive internationally and for the three systems listed in Table 3.13. However, this coefficient is much higher for Hong Kong and Australia, both at 0.22, which is also higher than the correlation coefficient between S_USESTD and CIL scores. This is not surprising as S_TSKLRN registers the learning of CIL specific tasks.

73

72(1

.4)

81

(1.4

)

66(1

.9)

53

(1.4

)

70(1

.7)

74

(1.4

)

71(1

.8)

64

(1.8

)48

(0.1

)0.

22(0

.0)

73(0

.3)

85

(0.2

)

76(0

.3)

70

(0.3

)

75(0

.3)

73

(0.3

)

72(0

.3)

67

(0.3

)

50(0

.1)

0.

08(0

.0)

87(0

.9)S

96(0

.4)S

92(0

.6)S

82(0

.8)S

91(0

.5)S

83(0

.8)S

77(0

.9)U

74(1

.0)U

54(0

.2)Ï

0.22

(0.0

)

70(1

.0)V

74(1

.0)T

60(1

.1)T

60(1

.0)V

60(1

.1)T

67(1

.1)V

59(1

.0)T

54(1

.1)T

46(0

.3)Ð

0.07

(0.0

)*

Perc

enta

ges

refle

ct s

tude

nts

that

sel

ecte

d "Y

es"

*St

atis

tical

ly s

igni

fican

t (p<

0.05

) coe

ffic

ient

s in

bol

d.( )

St

anda

rd e

rror

s ap

pear

in p

aren

thes

es. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

#M

ean

of s

cale

sco

re =

50S

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

abov

e IC

ILS

aver

age

ÏM

ore

than

3 s

core

poi

nts

abov

e IC

ILS

aver

age

USi

gnifi

cant

ly a

bove

ICIL

S av

erag

e×

Sign

ifica

ntly

abo

ve IC

ILS

aver

age

VSi

gnifi

cant

ly b

elow

ICIL

S av

erag

eØ

Sign

ifica

ntly

bel

ow IC

ILS

aver

age

TM

ore

than

10

perc

enta

ge p

oint

s be

low

ICIL

S av

erag

eÐ

Mor

e th

an 3

sco

re p

oint

s be

low

ICIL

S av

erag

e

Ove

rall

stud

ents

' av

erag

e sc

ale

scor

e fo

r le

arni

ng o

f ICT

ta

sks

at s

choo

l (S

_TSK

LRN)

#

Corr

elat

ion

coef

ficie

nts

for

CIL

with

S_

TSKL

RNPr

ovid

ing

refe

renc

es to

In

tern

et

sour

ces

Acce

ssin

g in

form

atio

n w

ith a

co

mpu

ter

Pres

entin

g in

form

atio

n fo

r a

give

n au

dien

ce o

r pu

rpos

e w

ith a

co

mpu

ter

Wor

king

out

w

heth

er to

trus

t in

form

atio

n fro

m th

e In

tern

et

Deci

ding

wha

t in

form

atio

n is

re

leva

nt to

in

clud

e in

sc

hool

wor

k

Org

anis

ing

info

rmat

ion

obta

ined

from

In

tern

et

sour

ces

Deci

ding

whe

re

to lo

ok fo

r in

form

atio

n ab

out a

n un

fam

iliar

topi

c

Look

ing

for

diffe

rent

type

s of

dig

ital

info

rmat

ion

on

a to

pic

Stud

ents

' lea

rnin

g of

ICT

task

s at

sch

ool t

hat a

re in

clud

ed in

sca

le s

core

S_T

SKLR

N

Educ

atio

nal S

yste

m

Hon

g K

ong

(SAR

)

ICIL

S 2

013

aver

age

Aust

ralia

Kor

ea, R

ep. o

f

Tabl

e 3.1

4 Pe

rcen

tage

s of s

tude

nts r

epor

ted

havi

ng le

arnt

CIL

rela

ted

task

s at s

choo

l, S_

TSKL

RN a

nd th

e ass

ocia

tion

with

CIL

scor

e

74

3.4.3 Use of computers outside school

There were four sets of questions in the student survey that collected information about  students’  ICT  use  outside  school,  one  of  which  was  about  students’  use  of  work-oriented ICT applications while the other three were about their use for various personal purposes. For all questions related to outside school use of ICT, students  were  given  five  response  options:  “never,”  “less  than  once  a  month,”  “at  least  once  a  month  but  not  every  week,”  “at  least  once  a  week  but  not  every  day,”  and  “every  day”  to  indicate their frequency of engagement. In the following four subsections, we report in table format the percentages of students engaging in each of the relevant type of computer use outside of school at least once a week. In addition, for each category of use, a scale was constructed using Rasch partial credit model with an international mean of 50 and standard deviation of 10. Work-oriented ICT applications (S_USEAPP) There  was  only  one  question  in  the  student  survey  that  asked  about  students’  use  of specific ICT applications, which were all work-oriented, and students were most likely to be using them for the purpose of doing schoolwork:

x Creating or editing documents (for example to write stories or assignments);

x Using a spreadsheet to do calculations, store data or plot graphs (for example using [Microsoft EXCEL®]);

x Creating   a   simple   “slideshow”   presentation   (for   example   using  [Microsoft PowerPoint®]);

x Creating a multi-media presentation (with sound, pictures, video); x Using education software that is designed to help with your school study

(for example mathematics or reading software); x Writing computer programs, macros or scripts (for example using [Logo,

Basic or HTML]); and x Using drawing, painting or graphics software. Table 3.15 presents the percentages of students reporting having used each

of these applications outside of school at least once a week. The results show that in nearly all countries, creating or editing documents for writing purposes was the one with the highest percentage of students reporting use at least once a week (the only exception was Lithuania), with an international mean of only 28%. In general, using programming languages to write applications was the least popular application, with 13 systems out of 21 registering less than 10% of students doing so at least once a week; and for Hong Kong, this percentage was only 8%. The second most popular type of ICT application used by students on a weekly basis outside of school was educational software designed to help with school studies, at 18% internationally and only 15% for Hong Kong.

75

26(1

.6)

11

(0.7

)

10(0

.8)

13

(0.8

)

15(1

.0)

8

(0.5

)

12(0

.7)

48(0

.4)

0.15

(0.0

)

28(0

.3)

11(0

.2)

17(0

.3)

15(0

.2)

18(0

.2)

10(0

.2)

18(0

.2)

50(0

.1)

0.09

(0.0

)

48(1

.3)S

9(0

.5)V

20(1

.0)U

15(0

.6)

28(1

.2)U

14(0

.7)U

19(0

.7)

52(0

.2)×

0.13

(0.0

)

13(0

.8)T

5(0

.4)V

5(0

.5)T

7(0

.5)V

11(0

.6)V

5(0

.5)V

8(0

.5)T

45(0

.3)Ð

0.22

(0.0

)*

Perc

enta

ges

refle

ct s

tude

nts

that

sel

ecte

d "A

t lea

st o

nce

a w

eek

but n

ot e

very

day"

& "E

very

day"

*St

atis

tical

ly s

igni

fican

t (p<

0.05

) coe

ffic

ient

s in

bol

d.( )

St

anda

rd e

rror

s ap

pear

in p

aren

thes

es. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

#M

ean

of s

cale

sco

re =

50S

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

abov

e IC

ILS

aver

age

ÏM

ore

than

3 s

core

poi

nts

abov

e IC

ILS

aver

age

USi

gnifi

cant

ly a

bove

ICIL

S av

erag

e×

Sign

ifica

ntly

abo

ve IC

ILS

aver

age

VSi

gnifi

cant

ly b

elow

ICIL

S av

erag

eØ

Sign

ifica

ntly

bel

ow IC

ILS

aver

age

TM

ore

than

10

perc

enta

ge p

oint

s be

low

ICIL

S av

erag

eÐ

Mor

e th

an 3

sco

re p

oint

s be

low

ICIL

S av

erag

e

Educ

atio

nal S

yste

m

Hon

g K

ong

(SAR

)

ICIL

S 2

013

aver

age

Aust

ralia

Kor

ea, R

ep. o

f

Corr

elat

ion

coef

ficie

nts

for

CIL

with

S_

USEA

PP

Stud

ents

' use

of c

ompu

ters

for s

peci

fic IC

T ap

plic

atio

ns th

at a

re in

clud

ed in

sca

le s

core

S_U

SEA

PP

Crea

ting

or

editi

ng

docu

men

ts (f

or

exam

ple

to

writ

e st

orie

s or

as

sign

men

ts)

Usin

g a

spre

adsh

eet t

o do

cal

cula

tions

, st

ore

data

or

plot

gra

phs

(for

exam

ple

usin

g [M

icro

soft

EXCE

L ®

])

Crea

ting

a si

mpl

e “slid

esho

w”  

pres

enta

tion

(for e

xam

ple

usin

g [M

icro

soft

Pow

erPo

int ®

])

Crea

ting

a m

ulti-

med

ia

pres

enta

tion

(with

sou

nd,

pict

ures

, vid

eo)

Usin

g ed

ucat

ion

softw

are

that

is

desi

gned

to

help

with

you

r sc

hool

stu

dy

(for e

xam

ple

mat

hem

atic

s or

re

adin

g so

ftwar

e)

Writ

ing

com

pute

r pr

ogra

ms,

m

acro

s or

sc

ripts

(for

ex

ampl

e us

ing

[Log

o, B

asic

or

HTM

L])

Usin

g dr

awin

g,

pain

ting

or

grap

hics

so

ftwar

e

Ove

rall

stud

ents

' av

erag

e sc

ale

scor

e fo

r use

of

com

pute

rs fo

r sp

ecifi

c IC

T ap

plic

atio

ns

(S_U

SEA

PP) #

Tabl

e 3.1

5 Pe

rcen

tage

s of s

tude

nts u

sing

wor

k-or

ient

ed a

pplic

atio

ns o

utsi

de sc

hool

at l

east

onc

e a w

eek,

S_U

SEAP

P an

d th

e ass

ocia

tion

with

CIL

scor

e

76

Similar to the other computer use statistics, Korea also reports significantly lower than the international average in terms of percentages of students reporting weekly outside school use of all seven ICT applications. For Australia, the trend was somewhat different compared to the previously reported use statistics, with only four applications showing significantly higher percentages than the international mean, and significantly lower than the international average for using a spreadsheet at least once a week. For Hong Kong, the mean percentages remain similar and slightly lower than the corresponding international means, except for the use of spreadsheets, which was the same as the international mean of 11%.

The scale reflecting level of use of these seven applications, S_USEAPP, had a  Cronbach’s  alpha  reliability  of  0.80.  The  scale  scores  for  Hong  Kong  and  Korea  were both lower than the international mean at 48 and 46 respectively, while the score for Australia was significantly higher at 52. The correlation coefficient between S_USEAPP and CIL score was also found to be positive for all three systems, with Korea having the highest correlation of 0.22, following by Hong Kong at 0.15 and Australia at 0.13. Use of ICT for exchanging information (S_USEINF) Students may also engage in information exchange activities outside of school for personal rather than study purposes. The survey included the following four activities belonging to this category:

x Asking questions on forums or [question and answer] websites; x Answering other peoples’  questions  on  forums  or  websites; x Writing posts for your own blog; and x Building or editing a webpage. Table 3.16 shows the percentages of students reporting using the Internet for

each of these activities at least once a week. It can be seen that the percentages in this Table is somewhat higher than those in Table 3.15, indicating that students are more likely to engage in these information exchange activities than in using work-oriented ICT applications, even though the percentages are still relatively low. Answering   other   peoples’   questions   on   forums   or   websites was the most popular information exchange activity internationally as well as in Hong Kong, at 24% and 30% of the student population respectively.

On the other hand, writing posts for your own blog was the most popular use in this category for Australian students, at 22%, while for Korean students it was asking questions on forums or [question and answer] websites at 18%. While the most popular information exchange activity differs across countries, building or editing a webpage was the least popular information exchange activity in all countries.

77

It is also interesting to note that unlike the other types of computer use

reported earlier, the percentages of Australian students reported engaging in the

three activities other than blogging were significantly lower than the international

mean.

Table 3.16 Percentages of students using the Internet outside school at least once a week for exchange of information, S_USEINF and the association with CIL score

Further, though the percentages of Korean students engaging in information

exchange activities outside school remain significantly lower than the international

average, their scale score for this category of use, S_USEINF, was in fact higher

than   Australia.   Hong   Kong’s   S_USEINF   score   was   similar   to   the   international  average, at 50. Another very surprising observation is that the correlation between

S_USEINF score and CIL score is in fact negative, though small, for Hong Kong,

Australia and the international mean.

Use of ICT for social communication (S_USECOM)

Using social media to communicate with others has become a very popular activity

many people use their computers to do. The student survey contained four items

of social media based communication activity to find out how often students

engage in this type of activity:

x Communicating with others using messaging or social networks (for

example instant messaging or [status updates]);

x Posting comments to online profiles or blogs;

x Uploading images or video to an [online profile] or [online community]

(for example Facebook or YouTube); and

x Using voice chat (e.g. Skype) to chat with friends or family online.

23 (1.0) 30 (0.9) 13 (0.8) 9 (0.8) 50 (0.2) -0.03 (0.0)

22 (0.3) 24 (0.3) 21 (0.2) 11 (0.2) 50 (0.1) -0.02 (0.0)

17 (0.8) V 13 (0.5) T 22 (0.7) 8 (0.5) V 48 (0.2) Ø -0.09 (0.0)

18 (0.8) V 16 (0.7) V 11 (0.6) T 5 (0.4) V 49 (0.1) Ø 0.07 (0.0)* Percentages reflect students that selected "At least once a w eek but not every day" & "Every day"* Statistically signif icant (p<0.05) coefficients in bold.

( ) Standard errors appear in parentheses. Because some results are rounded to the nearest w hole number, some totals may appear inconsistent. # Mean of scale score =50S More than 10 percentage points above ICILS average Ï More than 3 score points above ICILS averageU Signif icantly above ICILS average × Signif icantly above ICILS averageV Signif icantly below ICILS average Ø Signif icantly below ICILS averageT More than 10 percentage points below ICILS average Ð More than 3 score points below ICILS average

Correlation coefficients for

CIL with S_USEINF

Korea, Rep. of

Writing posts for your own

blog

Building or editing a webpage

Asking questions on

forums or [question and

answer] websites

Answering other  peoples’  questions on

forums or websites

Overall students'

average scale score for use of

ICT for exchanging information

(S_USEINF) #

Students' use of ICT for exchanging information that are included in scale score S_USEINF

Educational System

Hong Kong (SAR)

ICILS 2013 average

Australia

78

Table 3.17 shows the percentages of students using the Internet outside

school for social communication at least once a week were much higher than for

work-oriented applications or information exchange across the participating

countries. Internationally, the most popular social communication activity was

communicating with others using messaging or social networks, with an average of 65%,

followed by posting comments to online profiles or blogs at 49%.

Comparatively least popular was uploading images or video to an online

community such as Facebook or YouTube, which has an international average of

38%, and much higher than any of the information exchange activity. Students in

Hong Kong, Australia and Korea engaged in these activities somewhat less

frequently than the international average, as reflected by their scale scores for

social communication, S_USECOM, which were all below 50. Similar to most other

types of computer use, the correlation between S_USECOM score and CIL score

was positive, with the highest coefficient registered by Korea, at 0.16, followed by

Hong Kong at 0.12.

Table 3.17 Percentages of students using the Internet outside school at least once a week for social communication, S_USECOM and the association with CIL score

Use of ICT for recreation (S_USEREC)

Recreation is another important functional use of ICT by students and adults. The

survey asked students to indicate how often they use ICT outside school for the

following recreational activities:

x Accessing the Internet to find out about places to go or activities to do;

x Reading reviews on the Internet of things you might want to buy;

60 (1.6) 36 (1.3) 33 (1.1) 39 (1.3) 48 (0.3) � 0.12 (0.0)

75 (0.3) 49 (0.3) 38 (0.3) 48 (0.3) 50 (0.1) � 0.11 (0.0)

80 (0.8) U 48 (0.8) 36 (0.9) V 36 (1.0) T 49 (0.2) Ø 0.06 (0.0)

42 (1.1) T 35 (1.1) T 23 (0.9) T 26 (0.9) T 44 (0.2) Ð 0.16 (0.0)* Percentages reflect students that selected "At least once a w eek but not every day" & "Every day"* Statistically signif icant (p<0.05) coefficients in bold.

( ) Standard errors appear in parentheses. Because some results are rounded to the nearest w hole number, some totals may appear inconsistent. S More than 10 percentage points above ICILS average # Mean of scale score =50U Signif icantly above ICILS average Ï More than 3 score points above ICILS averageV Signif icantly below ICILS average × Signif icantly above ICILS averageT More than 10 percentage points below ICILS average Ø Signif icantly below ICILS average

Ð More than 3 score points below ICILS average

Australia

Korea, Rep. of

Uploading images or

video to an [online profile]

or [online community] (for

example Facebook or

Youtube)

Using voice chat (for example

Skype) to chat with friends or family online

Communicating with others

using messaging or

social networks (for example

instant messaging or

[status updates])

Posting comments to

online profiles or blogs

Correlation coefficients for

CIL with S_USECOM

Students' use of ICT for social communication purposes that are included in scale score S_USECOM

Educational System

Hong Kong (SAR)

ICILS 2013 average

Overall students'

average scale score for use of ICT for social

communication purposes

(S_USECOM) #

79

x Playing games;

x Listening to music;

x Watching downloaded or streamed video (for example movies, TV shows

or clips); and

x Using the Internet to get news about things I am interested in.

The results, as presented in Table 3.18, show that recreation is indeed a very

popular use of computers by students outside school. Listening to music is the single

most popular ICT-based recreational activity for all participating countries, with a

minimum percentage of 63% in Korea, and the highest in Denmark, at 92%. The

second most popular ICT-using recreation differs across countries, and was

generally either watching downloaded or streamed video, using the Internet to get news about things I am interested in, or playing games. The least popular ICT-using

recreation was accessing the Internet to find out about places to go or activities to do,

which technically is not recreation, but finding out information about recreation.

The level of ICT use by Hong Kong and Australian students for recreational

purposes were similar to the international average, both with a scale score of 50 for

S_USEREC. Korean students showed a lower level of engagement, with a scale

score of 48 only. S_USEREC is again  positively  correlated  with  students’  CIL  score.  However, the correlation coefficients differ greatly, with a very low and

insignificant one for Hong Kong, at 0.03, and the highest for Korea, at 0.15.

3.5  Students’  Contextual  Factors  and  Their  CIL  Achievement

While we can see from the foregoing sections that some of the student level

contextual factors are significantly correlated with their CIL scores, these factors

are often not independently of each other.

Further, as mentioned in section 3.1, it is important to note that there may

also be school effects on some of these variables collected through the student

survey. In this section we will explore how the totality of the key context variables

listed  in  Table  3.1   influence  students’ CIL achievement. To identify school level

effects, we have adopted multilevel analysis using HLM, even though all the

context variables are collected at one level through the student survey.

3.5.1 Contextual factors and overall CIL score

First of all, we will investigate the following two research questions for the Hong

Kong CIL results:

1. Do any of the context variables collected through the student survey

significantly influence their CIL achievement scores?

2. What percentage of the total between-school and within-school variance

can these variables account for?

80

29(1

.0)

30

(1.0

)

58(1

.3)

72

(1.0

)

64(1

.1)

68

(1.4

)50

(0.3

)0.

03(0

.0)

28(0

.3)

31(0

.3)

56(0

.3)

82(0

.2)

68(0

.3)

62(0

.3)

50(0

.1)

0.12

(0.0

)

31(0

.8)U

34(1

.1)U

55(1

.2)

80(0

.7)V

65(1

.1)V

51(1

.1)T

50(0

.2)

0.10

(0.0

)

25(0

.9)V

30(1

.0)

56(1

.3)

63(1

.0)T

54(1

.1)T

57(1

.1)V

48(0

.2)Ø

0.15

(0.0

)*

Perc

enta

ges

refle

ct s

tude

nts

that

sel

ecte

d "A

t lea

st o

nce

a m

onth

but

not

eve

ry w

eek"

& "A

t lea

st o

nce

a w

eek"

*St

atis

tical

ly s

igni

fican

t (p<

0.05

) coe

ffic

ient

s in

bol

d.( )

St

anda

rd e

rror

s ap

pear

in p

aren

thes

es. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

#M

ean

of s

cale

sco

re =

50S

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

abov

e IC

ILS

aver

age

ÏM

ore

than

3 s

core

poi

nts

abov

e IC

ILS

aver

age

USi

gnifi

cant

ly a

bove

ICIL

S av

erag

e×

Sign

ifica

ntly

abo

ve IC

ILS

aver

age

VSi

gnifi

cant

ly b

elow

ICIL

S av

erag

eØ

Sign

ifica

ntly

bel

ow IC

ILS

aver

age

TM

ore

than

10

perc

enta

ge p

oint

s be

low

ICIL

S av

erag

eÐ

Mor

e th

an 3

sco

re p

oint

s be

low

ICIL

S av

erag

e

Educ

atio

nal S

yste

m

Hon

g K

ong

(SAR

)

ICIL

S 2

013

aver

age

Aust

ralia

Kor

ea, R

ep. o

f

Corr

elat

ion

coef

ficie

nts

for

CIL

with

S_

USER

EC

Stud

ent's

use

of I

CT fo

r rec

reat

ion

purp

oses

that

are

incl

uded

in s

cale

sco

re S

_USE

REC

List

enin

g to

m

usic

Wat

chin

g do

wnl

oade

d or

st

ream

ed v

ideo

(fo

r exa

mpl

e m

ovie

s, T

V sh

ows

or c

lips)

Acce

ssin

g th

e In

tern

et to

find

ou

t abo

ut

plac

es to

go

or

activ

ities

to d

o

Read

ing

revi

ews

on th

e In

tern

et o

f th

ings

you

m

ight

wan

t to

buy

Play

ing

gam

es

Usin

g th

e In

tern

et to

get

ne

ws

abou

t th

ings

I am

in

tere

sted

in

Ove

rall

stud

ents

' av

erag

e sc

ale

scor

e fo

r use

of

ICT

for

recr

eatio

n pu

rpos

es

(S_U

SERE

C) #

Tabl

e 3.1

8 Pe

rcen

tage

s of s

tude

nts u

sing

the I

nter

net o

utsi

de sc

hool

at l

east

onc

e a w

eek

for r

ecre

atio

n, S

_USE

REC

and

the a

ssoc

iatio

n w

ith

CIL

scor

e

81

The first step in multilevel modelling is to conduct a variance component analysis to obtain information on the Null model regarding the total variances at the student and school levels respectively. The results show that under the null model, the within school (level 1) variance is 4009.67 and the between school (level 2) variance is 3929.95, which means that about half of the variance in Hong Kong students’  CIL  test  scores  relates  to  school  level  differences. Table 3.19 Multilevel  model  results  for  students’  CIL  scores  using  student  context  

variables as level 1 predictors

Category Coefficient Error p-valueINTRCPT Intercept 508.86 8.23 0.00

Students’  Personal  and Family Background

Z_ISCED Z: expected education by student 7.73 2.14 0.00

Z_ADVEFF Z: ICT self-efficacy advanced skills -8.81 3.81 0.03Z_BASEFF Z: ICT self-efficacy basic skills 25.67 3.34 0.00Z_INTRST Z: interest and enjoyment in using ICT - - -

Z_TSKLRN Z: learning of ICT tasks at school 5.52 2.49 0.03Z_USEAPP Z: use of specific ICT applications - - -Z_USELRN Z: use of ICT during lessons at school -5.37 2.50 0.04Z_USEREC Z: use of ICT for recreation - - -Z_USESTD Z: use of ICT for study purposes - - -Z_USECOM Z: use of ICT for social communication - - -Z_USEINF Z: use of ICT for exchanging information - - -

Between School Variance & Percentage 3925.95 49.5%

Within School Variance & Percentage 4009.67 50.5%

Between School 2846.80 27.5%

Within School 3555.52 11.3%

* All independent variables were transformed into zscores# The dependent variables (i.e. 5 plausible values) were in their original score format (Mean=509, SD=7.4 for HK sample)^ The dependent variable (i.e. CIL aspect scores) were presented in chapter 2.6. "Variance explained" refers to the percentage of variance the set of variables in the model has accounted for level 2 total variance."-" indicates variables that were not input into the specific model due to negligibly small effects on students' CIL scores (the outcome variable).Variables highlighted have p-value <=0.05

Variance explained by model and their percentages

Model: Student contextual factors and overall CIL Scores #

Students’  ICT  Self-­efficacy and Interest

Students’  ICT  Use  Experience at Home

and in School

Scale

Null Model

82

We then conducted Random Intercepts Fixed Slope modelling using all the

variables listed in Table 3.1 as level 1 predictor variables. The HLM results for the

analysis after reaching convergence is presented in Table 3.19. The results indicate

that only five variables remain statistically significant at the end of the modelling

process. Three of these have positive coefficients: Self-efficacy in basic ICT skills

(Z_BASEFF) has the highest coefficient of 25.67, followed by Educational

aspiration (ZS_ISCED, 7.73) and Learning of ICT tasks at school (ZS_TSKLRN,

5.52). The two factors that have negative coefficients are Self-efficacy in advanced

ICT skills (Z_ADVEFF, -.8.81) and Use of ICT in non-ICT lessons (Z_USELRN, -5.37).

The correlation of these two factors with CIL scores when computed

independently were relatively low, though not negative (as described in the

previous two sections). The negative coefficients indicate significant correlation

among the student context variables. It is also important to note that although

gender and SES variables were significantly correlated with CIL scores when

computed independently, they are no longer significant in the analysis results,

again due to their high correlation with other context variables.

Examining the variance explained by this model, we find that the between

school variance explained is much higher (at 27.5%) than the within school

variance explained (at 11.3%). This indicates that there is much similarity among

students within the same school in terms of their personal and family context as

well as their ICT use experience that contributes to explaining 27.5% of the between

school variance in students’  CIL   scores.  On   the   other  hand,   these   student   level  differences can only explain 11.3% of the variations in CIL scores within the same

school.

3.5.2 Contextual factors and the seven CIL aspect scores

In  chapter  2,  in  addition  to  reporting  on  students’  achievement in terms of their

overall CIL scores, we have also reported on their percentage scores for each of the

seven  CIL  aspects.  Furthermore,  we  find  that  Hong  Kong  students’  performance  was particularly poor in accessing and evaluating information, transforming

information, sharing information and managing information. Hence in this

section, we further employ Random Intercepts Fixed Slope modelling using HLM

to find out, for each CIL aspect.

1. Do any of the context variables collected through the student survey

significantly influence their CIL achievement scores?

2. What percentage of the total between-school and within-school variance

can these variables account for?

Since gender and SES dropped out from the multilevel model in very early

iterations in the previous analysis with the overall CIL score as the independent

variable, we have decided to remove these two from the current set of analyses,

83

with standardized z-scores of the percentages correct for each of the seven CIL aspects. Before discussing the results of the seven multilevel models presented in Table 3.20, we need to caution that the seven standardized CIL aspect scores differ greatly in the quality of their psychometric properties. This is primarily because the Study itself was designed to develop a single CIL achievement scale. The seven-aspects framework was used to guide the development of the assessment tasks, but the assessment design was not developed sufficiently to warrant a fully formed scale for each of the aspects, given the testing time is only limited to an hour.

A closely related limitation is that not all of the booklets include items from all seven aspects, which means the sample size for the multilevel analysis for some of the aspects could be smaller. This large variability in the quality of the seven standardized aspect scores is also reflected in the very large differences in distribution of total variance across between-school and within-school variance in the seven null models in Table 3.20. Keeping in mind the above cautions, the results reported here are only for exploratory purposes. There are nevertheless some interesting observations. First of all, consistent with the finding reported in section 3.4, we find that self-efficacy in basic ICT skills (Z_BASEFF) is significant for all seven models with a positive coefficient of at least 0.11, which is bigger than any of the other coefficients, indicating that this is a most important antecedent for students to develop their CIL skills. The second most important factor across the board   is   students’   educational   aspirations   (ZS_ISCED).   All   coefficients   for   this  factor are positive, and are statistically significant, except the aspect transforming information. Interest and enjoyment in using ICT (ZS_INTRST) is also a relatively important positive contributing factor, having positive coefficients for the aspects knowing about and understanding computer use, managing information, and transforming information. Learning of ICT tasks in school (ZS_TSKLRN) similarly has positive significant coefficients for three aspects: knowing about and understanding computer use, creating information and sharing information.

The factor with the third highest number of significant coefficients is self-efficacy in advanced ICT skills (Z_ADVEFF). However, similar to the findings reported in section 3.4, all of the coefficients are negative, and are significant for four aspects: accessing and evaluating information, transforming information, creating information, and using information securely and safely.

One prominent observation is that three types of reported use of ICT have not recorded a single instance of a significant coefficient in any of the CIL aspects. These are: use of ICT for recreation (Z_USEREC), use of work-oriented ICT applications (Z_USEAPP) and use of ICT for study purposes (Z_USESTD). Apparently, ICT use at school and for study purposes have not contributed much to   students’   CIL   achievement. Similar to the multilevel analysis results for the overall   CIL   score,   students’   context   variables can only explain a substantial percentage of the between-school variance but not the within-school variance in student achievement for all seven CIL aspects.

84

Ta

ble 3

.20

Mul

tilev

el m

odel

resu

lts fo

r eac

h of

the s

even

stan

dard

ized

CIL

asp

ect s

core

s usi

ng st

uden

t con

text

var

iabl

es a

s lev

el 1

pre

dict

ors

Va

riabl

esB

rief d

escr

iptio

n

Coe

ffici

ent

Erro

rp-

valu

eC

oeffi

cien

tEr

ror

p-va

lue

Coe

ffici

ent

Erro

rp-

valu

eC

oeffi

cien

tEr

ror

p-va

lue

Coe

ffici

ent

Erro

rp-

valu

eC

oeffi

cien

tEr

ror

p-va

lue

Coe

ffici

ent

Erro

rp-

valu

eIN

TRC

PT1

-0.0

40.

050.

36-0

.10

0.06

0.09

-0.0

60.

040.

10-0

.09

0.06

0.15

-0.1

20.

080.

12-0

.05

0.03

0.14

-0.0

50.

060.

36Z_

ISC

EDZ:

exp

ecte

d ed

ucat

ion

by s

tude

nt0.

090.

030.

010.

060.

030.

010.

060.

030.

020.

040.

020.

130.

090.

020.

000.

090.

020.

000.

060.

030.

03Z_

ADVE

FFZ:

ICT

self-

effic

acy

adva

nced

ski

lls-

--

-0.1

10.

030.

00-0

.05

0.04

0.15

-0.1

50.

030.

00-0

.10

0.03

0.00

-0.0

40.

040.

25-0

.10

0.03

0.01

Z_BA

SEFF

Z: IC

T se

lf-ef

ficac

y ba

sic

skills

0.17

0.03

0.00

0.23

0.03

0.00

0.12

0.04

0.00

0.25

0.03

0.00

0.24

0.03

0.00

0.11

0.03

0.00

0.19

0.04

0.00

Z_TS

KLR

NZ:

lear

ning

of I

CT

task

s at

sch

ool

0.06

0.03

0.04

0.03

0.03

0.19

0.02

0.03

0.45

0.03

0.03

0.20

0.08

0.02

0.00

0.08

0.02

0.00

0.03

0.03

0.25

Z_U

SEAP

PZ:

use

of s

peci

fic IC

T ap

plic

atio

ns0.

060.

030.

060.

030.

030.

22-

--

0.05

0.03

0.15

--

-0.

020.

030.

530.

020.

040.

63Z_

USE

LRN

Z: u

se o

f IC

T du

ring

less

ons

at s

choo

l-0

.04

0.02

0.12

-0.0

90.

030.

00-0

.07

0.03

0.01

-0.0

30.

030.

34-0

.02

0.02

0.33

-0.0

40.

020.

07-0

.01

0.03

0.64

Z_U

SER

ECZ:

use

of I

CT

for r

ecre

atio

n-

--

--

--

--

-0.0

30.

030.

35-0

.04

0.02

0.10

-0.0

10.

030.

79-0

.02

0.03

0.48

Z_U

SEST

DZ:

use

of I

CT

for s

tudy

pur

pose

s-

--

0.04

0.03

0.15

--

-0.

000.

020.

85-

--

0.00

0.02

0.84

0.05

0.03

0.17

Z_U

SEC

OM

Z: u

se o

f IC

T fo

r soc

ial c

omm

unic

atio

n0.

050.

040.

20-

--

0.05

0.03

0.05

0.05

0.04

0.15

0.08

0.03

0.01

0.06

0.05

0.23

0.06

0.04

0.18

Z_IN

TRST

Z: in

tere

st a

nd e

njoy

men

t in

usin

g IC

T0.

070.

030.

03-

--

0.09

0.03

0.00

0.05

0.02

0.04

0.03

0.02

0.27

-0.0

10.

020.

730.

010.

020.

75Z_

USE

INF

Z: u

se o

f IC

T fo

r exc

hang

ing

info

rmat

ion

-0.0

30.

030.

28-0

.04

0.02

0.07

-0.0

90.

030.

00-0

.01

0.04

0.78

-0.0

50.

030.

05-0

.09

0.04

0.03

--

-

Betw

een

Scho

ol V

aria

nce

& Pe

rcen

tage

0.13

0011

.7%

0.29

3529

.6%

0.07

727.

9%0.

2970

29.1

%0.

4299

39.9

%0.

0709

7.2%

0.15

8315

.5%

With

in S

choo

l Var

ianc

e &

Perc

enta

ge0.

9772

88.3

%0.

6989

70.4

%0.

8994

92.1

%0.

7249

70.9

%0.

6471

60.1

%0.

9100

92.8

%0.

8634

84.5

%

Betw

een

Scho

ol0.

0585

54.9

7%0.

1830

37.6

6%0.

0324

58.0

5%0.

1887

36.4

8%0.

2825

34.2

9%0.

0288

59.4

0%0.

1060

33.0

6%

With

in S

choo

l0.

8863

9.30

%0.

6746

3.47

%0.

9000

-0.0

7%0.

6966

3.91

%0.

5924

8.44

%0.

9007

1.02

%0.

8350

3.29

%

* All

inde

pend

ent v

aria

bles

wer

e tra

nsfo

rmed

into

zsc

ores

# Th

e de

pend

ent v

aria

bles

(i.e

. 5 p

laus

ible

val

ues)

wer

e in

thei

r orig

inal

sco

re fo

rmat

(Mea

n=50

9, S

D=7

.4 fo

r HK

sam

ple)

^ Th

e de

pend

ent v

aria

ble

(i.e.

CIL

asp

ect s

core

s) w

ere

pres

ente

d in

cha

pter

2.6

. "V

aria

nce

expl

aine

d" re

fers

to th

e pe

rcen

tage

of v

aria

nce

the

set o

f var

iabl

es in

the

mod

el h

as a

ccou

nted

for l

evel

2 to

tal v

aria

nce.

"-" i

ndic

ates

var

iabl

es th

at w

ere

not i

nput

into

the

spec

ific

mod

el d

ue to

neg

ligib

ly s

mal

l effe

cts

on s

tude

nts'

CIL

sco

res

(the

outc

ome

varia

ble)

.Va

riabl

es h

ighl

ight

ed h

ave

p-va

lue

<=0.

05

Null

Mod

el

Varia

nce

expl

aine

d b

y m

odel

and

thei

r pe

rcen

tage

s

Mod

el: S

tude

nt c

onte

xtua

l fac

tors

and

the

7 CI

L as

pect

sco

res^

Aspe

ct 1

.1

Kno

win

g ab

out a

nd

unde

rsta

ndin

g co

mpu

ter u

se

Aspe

ct 1

.2

Asse

ssin

g &

Ev

alua

ting

Info

rmat

ion

Aspe

ct 1

.3

Man

agin

g In

form

atio

n

Aspe

ct 2

.1

Tran

sfor

min

g In

form

atio

n

Aspe

ct 2

.2

Cre

atin

g in

form

atio

nAs

pect

2.3

Sh

arin

g In

form

atio

n

Aspe

ct 2

.4

Usi

ng in

form

atio

n se

cure

ly &

saf

ely

85

3.6 Summary

In this chapter, we have reported on all the contextual variables collected from the student survey in the ICILS study, and investigated how these variables are associated  with  students’  CIL  scores.  When  considered  individually,  nearly  all  of  the contextual variables show a positive correlation with the CIL scores. The only exceptions were S_USELRN, the scale score for ICT use during lessons and S_USEINF, the scale score for ICT use outside school for information exchange. Both of these variables show a negative correlation. The list of student context scale variables and their corresponding correlation coefficients are presented in Table 3.21. Table 3.21 List of key student context scale variables and their correlation with CIL scores for Hong Kong

While these context variables are highly correlated and multilevel analyses

were subsequently conducted and reported in the previous two sections, the simple correlations are still worth consideration. First of all, it is clear that the two most   important   context   variables   influencing   Hong   Kong   students’   CIL  competence are their self-efficacy in basic ICT skills and opportunities to learn CIL-related tasks at school, with correlation coefficients of 0.40 and 0.22 respectively. In   comparison,   students’   use   of   ICT   outside   school   for   recreation   and   for  exchanging information have negligible association with their CIL competence.

Results of the multilevel  modelling  of  the  effect  of  students’  context  variables  on CIL scores as the independent variable corroborate the primary importance of students’  self-efficacy in basic ICT skills, with a coefficient of 25.67. Opportunities to learn CIL-related tasks  at  school  and  the  students’  educational  aspirations  also  have  significant  influence  on  students’  CIL  scores,  with  coefficients  at  5.52  and  7.73  respectively. However, both self-efficacy in advanced ICT skills and use of ICT during lessons at school had significant negative coefficients of -8.81 and -5.37 respectively.

Context scale variables Description Correlation with CIL

S_BASEFF Self-efficacy in basic ICT skills 0.4S_ADVEFF Self-efficacy in advanced ICT skills 0.09S_INTRST Interest and enjoyment in using ICT 0.12S_TSKLRN Learning of CIL-related tasks at school 0.22S_USEAPP Use of work-oriented ICT applications 0.15S_USELRN Use of ICT during lessons at school -0.13S_USESTD Use of ICT for study purposes 0.11S_USEREC Use of ICT for recreation 0.03S_USECOM Use of ICT for social communications 0.12S_USEINF Use of ICT for exchanging information -0.03

86

These  findings  tell  us  that  Hong  Kong  students’  use  of  ICT  in  lessons  and  for  study-related purposes are relatively low compared to their international counterparts, and that such experiences have not contributed positively to their CIL development.

Further  multilevel  analysis  of  the  influence  of  students’  context  variables  on  each of the seven CIL aspect scores show similar findings as for that of the overall CIL score. These findings are valuable in informing us of how schools can help students enhance their CIL competence. The teaching of CIL-related tasks in school, and helping students to develop basic ICT skills efficacy will contribute to enhancing CIL competence.

Another important finding   is   that   students’   SES   index   had   no   significant  effect on their CIL scores in the multilevel analysis results. Furthermore, all the student context variables combined only explained 11.3% of the within school variance   in   students’   CIL   scores,   but   explained 27.5% of the between school variance. This, coupled with the fact that Hong Kong has among the largest between school variance in student CIL achievement, indicates that the school a student attends has an extremely important association with the student’s   CIL  score. In the next section, we will explore the school effects through analysing the data collected from the school and teacher questionnaires.

87

Chapter 4

How Do Schools Influence Students’ CIL Achievement?

Apart from home, school is the most important venue where students develop their knowledge and understanding of the world. Compared to home, the school environment is more prone to changes, driven by the availability of human capital or financial resources, as well as territory wide education policies. Given the significance of the school environment, this chapter sets up to investigate the influence  of  school  level  factors  on  students’  CIL  achievement.

School  level  factors  that  influence  students’  achievement  can  be  categorized  into direct or indirect ones. Direct factors are those that contribute directly to students’   development   of   CIL,   such   as   the   overall   quality   of   the   learning  environment,  including  the  availability  of  ICT  resources  and  support,  and  students’  learning opportunities.  Indirect  factors  are  those  that  influence  students’  learning  outcomes through their effect on the direct factors. An example of an indirect factor is the leadership characteristics of the principal, who makes important decisions on infrastructure development and resource allocation, as well as the curriculum goals and strategic priorities.

In ICILS 2013, in each of the sampled schools, the school principal, ICT coordinator, and 15 randomly sampled teachers who teach grade 8 classes were invited to complete a questionnaire. The school principal questionnaire collected information  about  the  principals’  views  and  management  practices  regarding  ICT  use in their schools. The ICT coordinator questionnaire collected information about the ICT resources available and the obstacles encountered in implementing ICT use   in   the   school.  The   teacher  questionnaire  gathered  data  on   the   teachers’   ICT  background as well as their views and practices related to ICT in teaching and learning. Sampling of teachers and students in the same sampled school was conducted independently, and no data was collected on the specific grade 8 classes that the surveyed teachers taught in their school. Hence, data collected from the teacher questionnaire cannot be directly linked to the students. This implies that there cannot be any relational or causal analysis that connects these two sets of data at the individual level. Instead, the teacher questionnaire data collected from each  school  is  aggregated  such  that  the  teachers’  responses  would  serve as school level indicators. A total of 1338 grade 8 teachers and 115 principals participated in the ICILS-HK study. The overall participation rate after replacement and weighting for teachers was 58.3%, which was, unfortunately, lower than the IEA

88

requirement of 75%. Hence, similar to the results from the Hong Kong student data,

the school level findings for Hong Kong are marked as of category 2 quality and

were not included in the calculation of the ICILS international averages presented

in the international report published by the IEA.

In this chapter, we will first report on the condition of schools in Hong Kong

with regard to the three categories of direct and indirect factors: (1) school level

infrastructure and resource provisions, (2) principals’   e-Learning leadership

practices,   and   (3)   teachers’   ICT-using pedagogy. Similar to the strategy of our

reporting in Chapter 3, corresponding statistics for the international, Australian,

and South Korean data collected in ICILS 2013 are presented here to provide

relevant reference for comparative purposes.

In  order  to  explore  how  the  various  school  level  factors  influence  students’  CIL outcomes, we need to derive reliable scales for these factors from the

questionnaire returns. We have not used the scales developed by the IEA for this

purpose as school contexts differ greatly across countries, and hence the factor

structure  and  the  relationship  between  school  factors  and  students’  CIL  outcomes  would probably be very different. Since we are primarily interested in exploring

these for Hong Kong, we have conducted confirmatory factor analysis on the data

collected from the principal and teacher questionnaires to identify reliable scales

for use for multilevel modelling in section 4.4. The specific factors we have

identified and used in the multilevel modelling are also reported together with the

reporting of the related descriptive statistics.

Based on findings from the multilevel analysis, we will discuss at the end of

this chapter how the school level factors influence both the overall CIL score as

well as the scores for each of the seven CIL aspects of Hong Kong students.

4.1 ICT infrastructure and resources in schools

Data from the ICT coordinator questionnaire provide an overall picture of the ICT

infrastructure and resources available in the schools in Hong Kong and of how

these compare with those in other countries. In addition to the level of resources

available, the data also reflect the allocation and arrangement of ICT resources in

schools.

4.1.1 Digital learning resources

Some of the digital learning resources are web-based, while others are not.

Table 4.1 shows the percentages of students studying at schools where the

following internet-related teaching and learning resources were available:

x Computer-Based Information Resources (e.g., Wikis, Encyclopedias,

Websites,);

x Interactive Digital Learning Resources (e.g., Learning Objects);

x Access to the World Wide Web;

89

x Access to an Education Site or Network Maintained by an Education System;

x Email Accounts for Teachers; and x Email Accounts for Students.

Table 4.1 Percentages of students studying at schools with available internet-related teaching and learning resources

It can be seen from Table 4.1 that the percentages in Hong Kong are higher

than the international mean in all of the above provisions, at above 90%, except for email accounts for students (89%), and much higher than Korea in the provisions of interactive digital learning resources, email accounts for teachers and email accounts for students. The status of non-Internet based software resources for learning was similar. Table 4.2 shows the percentages of students studying at schools where the following software resources were available:

x Tutorial Software or [Practice Programs]; x Digital Learning Games; x Wordprocessing, Databases, Spreadsheets (e.g., Microsoft© Office Suite); x Multimedia Production Tools (e.g., Media Capture and Editing, Web

Production); x Data-Logging and Monitoring Tools; x Simulations and Modeling Software; x Presentation Software (e.g. Microsoft PowerPoint®, Keynote ®); x Communication Software (e.g., Email, Chat, Blogs, Other Social Media);

and x Graphing or Drawing Software.

83 (4.0) 65 (5.5) 64 (6.0) 86 (3.2) 84 (3.8) 38 (5.3)81 (1.2) 56 (1.4) 64 (1.5) 82 (1.2) 82 (1.3) 53 (1.5)79 (3.2) 85 (2.5) S 93 (1.7) S 95 (1.5) S 94 (1.5) S 73 (6.3) S

81 (5.6) 78 (5.2) S 79 (5.0) S 85 (4.1) 87 (3.7) 68 (7.1) S

( )

S More than 10 percentage points above ICILS averageU Significantly above ICILS averageV Significantly below ICILS averageT More than 10 percentage points below ICILS average

Standard errors appear in parentheses. Because some results are rounded to the nearest whole number, some totals may appear inconsistent.

Developing students' computer

skills, such as word

processing, spreadsheet operations, and email

Developing collaborative

and organisational

skills

Using ICT for facilitating students’  

responsibility for their own

learning

Using ICT to augment and

improve students’  learning

Developing students’  

understanding and skills relating to safe and

appropriate use of ICT

Developing students’  

proficiency in accessing and

using information

with ICT

Educational System

Hong Kong (SAR)ICILS 2013 averageAustraliaKorea, Rep. of

90

Tabl

e 4.2

Per

cent

ages

of s

tude

nts s

tudy

ing

at sc

hool

s with

ava

ilabl

e sof

twar

e res

ourc

es fo

r tea

chin

g an

d/or

lear

ning

91(3

.5)

65

(4.9

)

100

(0.0

)

100

(0.0

)

83(4

.1)

63

(5.3

)

100

(0.0

)

94(2

.8)

98

(1.4

)88

(0.7

)76

(0.8

)98

(0.3

)80

(0.8

)54

(1.1

)41

(1.0

)99

(0.2

)91

(0.6

)87

(0.7

)92

(2.2

)95

(1.7

)S

100

(0.0

)U

99(0

.3)S

85(2

.4)S

85(2

.8)S

100

(0.0

)U

98(1

.0)U

99(0

.6)S

88(2

.5)

78(3

.5)

98(1

.1)

87(3

.0)U

56(4

.4)

38(4

.0)

99(0

.9)

94(1

.9)

89(2

.6)

*P

erce

ntag

es re

flect

teac

hers

that

sel

ecte

d "I

n ev

ery

or a

lmos

t eve

ry le

sson

" & "I

n m

ost l

esso

ns"

( )

Sta

ndar

d er

rors

app

ear i

n pa

rent

hese

s. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

SM

ore

than

10

perc

enta

ge p

oint

s ab

ove

ICIL

S a

vera

geU

Sig

nific

antly

abo

ve IC

ILS

ave

rage

VS

igni

fican

tly b

elow

ICIL

S a

vera

geT

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

belo

w IC

ILS

ave

rage

Educ

atio

nal

Syst

emH

ong

Kon

g (S

AR)

ICIL

S 2

013

aver

age

Aust

ralia

Kor

ea, R

ep. o

f

Pres

enta

tion

softw

are

(e.g

. [M

icro

soft

Pow

erPo

int

®],

[Key

note

®

])

Com

mun

icat

ion

softw

are

(e.g

. em

ail,

chat

, blo

gs,

othe

r soc

ial

med

ia)

Gra

phin

g or

dr

awin

g so

ftwar

e

Tuto

rial

softw

are

or

[pra

ctic

e pr

ogra

ms]

Digi

tal

lear

ning

ga

mes

Wor

d-pr

oces

sing

, da

taba

ses,

sp

read

shee

ts

(e.g

. [M

icro

soft©

of

fice

suite

])

Mul

timed

ia

prod

uctio

n to

ols

(e.g

. m

edia

ca

ptur

e an

d ed

iting

, web

pr

oduc

tion)

Data

-lo

ggin

g an

d m

onito

ring

tool

s

Sim

ulat

ions

an

d m

odel

ling

softw

are

91

Hong Kong has a higher percentage of provision than the international and

the Korean means in all software resource categories, except digital learning games,

for which only 65% of HK students reported to have access to in their schools, as

compared with 76% and 78% by their international and Korean peers respectively.

In addition, provisions are much higher for Hong Kong students for data-logging

and monitoring tools, as well as simulations and modeling software, at 83% and

63% respectively, compared with the corresponding international means of 54%

and 41%.

4.1.2 Computer resources for teaching and/or learning

Besides digital teaching and learning resources, network infrastructure to support

teaching and learning activities as well as devices to support mobile learning (e.g.,

tablets) may also have significant influence on the implementation of e-Learning

in schools. The ICT coordinator survey also elicited information about the

availability of the following resources, and the results of which are presented in

Table 4.3:

x Access to a Local Area Network (LAN) in the School;

x Tablet devices (e.g., iPad and similar);

x Space on a school network for students to store their work;

x A school intranet with applications and workspaces for students to use

(e.g., Moodle);

x Internet-based applications for collaborative work (e.g., Google Docs®);

x A Learning Management System (e.g., WebCT®).

Table 4.3 Percentages of students at schools with computer resources for teaching and/or learning

Unlike the other categories related to ICT resources, Korea has higher levels

of provision than HK in three of the six areas: tablet devices, Internet-based

applications for collaborative work and Learning Management Systems.

99 (0.7) 36 (5.4) 95 (2.3) 90 (3.3) 52 (5.6) 65 (5.3) 94 (0.5) 19 (0.8) 65 (1.0) 37 (1.0) 46 (1.0) 35 (0.8)

100 (0.1) U 64 (3.7) S 98 (0.9) S 83 (2.5) S 67 (3.1) S 77 (2.8) S

93 (2.3) 48 (3.7) S 39 (4.1) T 69 (3.8) S 62 (4.1) S 94 (2.0) S

( )

S More than 10 percentage points above ICILS averageU Significantly above ICILS averageV Significantly below ICILS averageT More than 10 percentage points below ICILS average

Standard errors appear in parentheses. Because some results are rounded to the nearest whole number, some totals may appear inconsistent.

Access to a local area

network (LAN) in the school

Tablet devices (e.g. [iPad] and similar)

Space on a school

network for students to store their

work.

A school intranet with applications

and workspaces for students to use (e.g. [Moodle])

Internet-based applications

for collaborative

work (e.g. [Google Docs®])

A learning management system (e.g. [WebCT®])

Educational System

Hong Kong (SAR)ICILS 2013 averageAustraliaKorea, Rep. of

92

In particular, 94% of Korean students attended schools that provide LMS, which is much higher even when compared to the already high Australian mean of 77%. This reflects a stronger emphasis on using ICT in learning activities in Korean schools compared with Hong Kong.

4.1.3 Student: Computer ratios

In the ICT coordinator survey, computers refer to desktops, laptops, netbooks, tablets as well as terminals with a keyboard and a screen, but not including those used as a server. Internationally, for urban schools, the mean was 20. The highest provision was reported in Norway, with a ratio of 2, followed by Australia, with a ratio of 3; while the lowest ratio was found in Turkey, at 80. Hong Kong schools had an above average ratio of 8, while the figure for Korea was 20, the same as the international average. Percentages of students at schools with school computers at different locations In the ICT coordinator survey, the percentages of students with school computers at the following locations were explored:

x In Most Classrooms (80% or More). x In Computer Laboratories. x As Class Sets of Computers That Can Be Moved Between Classrooms x In the Library. x In Other Places Accessible to Students (e.g., Cafeteria, Auditorium, Study

Area); and x Student Computers (School-Provided or Student-Owned) Brought by

Students to Class.

From the figures in Table 4.4, we can see that Hong Kong is higher than the international mean in all of these different types of computer provisions except for “Student   computers   brought   by   students   to   class”   (11%) as compared with the international mean of 18%.

Across all the schools that participated in ICILS 2013, computer laboratories is the location where most students (95%) have access to computers in schools, followed by the school library (64%). The percentage of students with access to computers that can be moved between classrooms was 34%, and those in schools that had computers located in most classrooms amounted to 33%. The lowest percentages in terms of the type of access were in other places in school (17%) and students bringing their own computers to the school (18%).

Obviously, there were wide variations across the countries, and the overall international means may not be representative of the situations in individual countries.

93

Table 4.4 Percentages of students at schools with school computers at different locations

As mentioned, the library was the second most common location, after

computer laboratories, where most students could gain access to computers in schools (64%). In Hong Kong, 93% of students could access computers in libraries, which is the highest system average in the Study. The percentage of students who could access computers in the school library was 90% in Australia and 80% in Korea.

Hong Kong also has the highest percentage of students who are in schools with computers set up in most classrooms, at 84%. In contrast, the corresponding figures for Australia and Korea were only 20% and 40%, respectively. Instead of installing computers in most classrooms, which are most likely to be machines for teacher presentation, more than half of the Australian Grade 8 students (58%) had access to sets of computers that can be moved between classrooms. Such mobile sets of computers allow students to engage in hands-on e-learning activities. In Korea, the percentage of students with access to mobile classroom computer sets was 41%, similar to the percentage with computers in most classrooms in the school (40%). The between-system variations in access were greatest in the percentage of students whose school allows students to bring their own computers to their schools. This figure was highest in Western countries: Australia (53%), Norway (48%), Denmark (83%), and Ontario (56%), as well as Newfoundland and Labrador (52%) in Canada. The corresponding figures for Hong Kong and Korea were much lower, at 11% and 4%, respectively.

4.1.4 Summary

HK students had relatively good access to computers and the Internet at school for instructional purposes: 100% had a computer lab and 84% had computers available in most classrooms in their schools. There was also no relative lack of digital learning resources for students in Hong Kong.

However, computers that students could access for e-learning at school, e.g. through class sets of computers that can be moved between classrooms or on

100 (0.0) 93 (2.7) 32 (4.8) 84 (4.3) 34 (5.3) 11 (3.4) 95 (0.4) 64 (0.9) 34 (1.0) 33 (1.0) 17 (0.8) 18 (0.8)85 (2.5) V 90 (2.1) S 58 (3.7) S 20 (2.6) T 24 (3.0) U 53 (3.9) S87 (2.5) V 80 (3.4) S 41 (4.2) 40 (3.3) U 21 (3.2) 4 (1.6) T

( )

S More than 10 percentage points above ICILS averageU Significantly above ICILS averageV Significantly below ICILS averageT More than 10 percentage points below ICILS average

Standard errors appear in parentheses. Because some results are rounded to the nearest whole number, some totals may appear inconsistent.

Student computers

(school-provided or

student-owned)

brought by students to

class

In most classrooms

(80% or more)

In computer laboratories

As class sets of computers that can be

moved between

classrooms

In the library

In other places

accessible to students (e.g.

cafeteria, auditorium, study area)

Educational System

Hong Kong (SAR)ICILS 2013 averageAustraliaKorea, Rep. of

94

computers brought by the student to class, were relatively low. In terms of network infrastructures to support learning, HK was comparatively lower on internet-based applications for collaborative work, and access to a learning management system (65%) as compared with Korea (94%).

4.2 School Policies and Practices Regarding ICT Use

In the principal questionnaire, there were four main categories of data collected: 1) the principal’s   personal   background   and   his/her   personal   use   of   ICT, 2) the school’s  ICT  demographics,  3)   the  school’s  policies  and  practices  with  regard  to  ICT use for teaching and learning and 4) the management of ICT in the school. Preliminary analyses reveal that the most important factors are those related to leadership practices in schools. As mentioned at the beginning of this chapter, we have identified some scales that are of interest for further exploration based on the principals’   questionnaire   responses   on   e-learning leadership practices. In this section, we will report the findings on the principals’  views  about  ICT  use  in  their  schools, as well as the policies and practices that they have initiated in coherent groups of questionnaire items. Where the group of items give rise to a scale score, we will put the variable name of the corresponding scale score in the title of the related subsection. However, the scale scores would not be further discussed as a descriptive statistic, and will only be explored further in section 4.4 as an integral part of the multilevel analysis results.

4.2.1  Principals’  views  on  educational  purposes  of  ICT  use

Principals were asked to indicate the importance of ICT use for achieving each of the following outcome goals, using a 3-point Likert scale, namely, not important, somewhat important, or very important:

x Developing students' computer skills, such as word processing, spreadsheet operations, and email;

x Using ICT for  facilitating  students’  responsibility  for  their  own  learning; x Using  ICT  to  augment  and  improve  students’  learning; x Developing   students’   understanding   and   skills   relating   to   safe   and  

appropriate use of ICT; x Developing  students’  proficiency  in  accessing  and using information with

ICT; and x Developing collaborative and organisational skills.

Hong Kong principals gave the highest priority to the three skill-oriented

outcomes (all >80%): (1) basics skills in using the office suite of applications and email, (2) proficiency in accessing and using information, and (3) safe and appropriate use of ICT.

95

These percentages are similar to the international means profile. Goals

related  to  improving  students’  general  learning  outcomes  and  fostering  students’  responsibility for their own learning were considered very important by only 64%

and 65%, respectively. This contrasts strongly with the Australian principals (93%

and 85%) and Korean principals (79% and 78%). Only 38% of Hong Kong students

attended schools whose principals consider ICT use to be very important for

developing   students’   collaborative   and   organizational   skills,   whereas   the  international mean was 53%, and the Australian and Korean means were even

higher, at 73% and 68%, respectively. It is thus evident that  Hong  Kong  principals’  views on the role of ICT for learning and teaching were still focused on traditional

outcomes, and gave much lower priority to ICT use in fostering 21st century skills.

Table 4.5 Percentages of students at schools where the principals consider ICT use as very important for achieving different educational outcomes

4.2.2  Principals’  expectations  of  teachers’  knowledge  and  skills  in  professional use of ICT (P_EXPTPK)

What are the ICT-related knowledge and skills that principals expect and/or

require teachers in their schools to possess? The principals were asked to indicate

which one of three responses (not expected, expected but not required, expected

and required) applied to the following two categories of skills:

(a) Communication skills:

x Communicating with other staff via ICT ;

x Collaborating with other teachers via ICT; and

(b) Technological pedagogical content knowledge (TPCK)

x Integrating Web-based learning in their instructional practice;

x Using ICT-based forms of student assessment;

x Using ICT for monitoring student progress;

83 (4.0) 65 (5.5) 64 (6.0) 86 (3.2) 84 (3.8) 38 (5.3)81 (1.2) 56 (1.4) 64 (1.5) 82 (1.2) 82 (1.3) 53 (1.5)79 (3.2) 85 (2.5) S 93 (1.7) S 95 (1.5) S 94 (1.5) S 73 (6.3) S

81 (5.6) 78 (5.2) S 79 (5.0) S 85 (4.1) 87 (3.7) 68 (7.1) S

( )

S More than 10 percentage points above ICILS averageU Significantly above ICILS averageV Significantly below ICILS averageT More than 10 percentage points below ICILS average

Standard errors appear in parentheses. Because some results are rounded to the nearest whole number, some totals may appear inconsistent.

Developing students' computer

skills, such as word

processing, spreadsheet operations, and email

Developing collaborative

and organisational

skills

Using ICT for facilitating students’  

responsibility for their own

learning

Using ICT to augment and

improve students’  learning

Developing students’  

understanding and skills relating to safe and

appropriate use of ICT

Developing students’  

proficiency in accessing and

using information

with ICT

Educational System

Hong Kong (SAR)ICILS 2013 averageAustraliaKorea, Rep. of

96

x Integrating ICT into teaching and learning; x Using subject-specific learning software (e.g. tutorials, simulations); x Using e-portfolios for assessment; and x Using ICT to develop authentic (real-life) assignments for students.

Table 4.6 presents the percentages of students whose principals expect and

require their teachers to have the above skills. Interestingly, in Hong Kong and Australia, the TPCK that principals most cared about was the ability to use ICT to communicate with other staff, at 86% and 82% respectively, which is much higher than the use of ICT to collaborate with other teachers. In comparison, Hong Kong principals had relatively low expectations of teachers’  TPCK  in  other  areas,  with  the highest being the integration of ICT into teaching and learning, with a percentage of only 68%. The lowest expectations were related to the use of ICT for assessment and monitoring of students, none of which had a percentage higher than 30%. In particular, the percentage of students whose principals expected teachers to be able to use ICT to develop authentic (real-life) assignments for students was only 16%. This is very disappointing as bringing authentic contexts into the classroom is one of the potential strengths that ICT use could offer.

4.2.3  The  extent  to  which  principals  monitored  teachers’  ICT  use  to  achieve different learning outcomes

Principals   were   asked   whether   and   how   they   monitor   teachers’   ICT   use   for  students’ learning  or  for  developing  students’  advanced  ICT  skills.   Monitoring   of   teachers’   ICT   use   for   more   general   aspects   of   student   learning (P_MONSULRN) The  questionnaire  asked  principals  whether  they  monitor  teachers’  ICT  usage  for  the following aspects of students’  learning:  

x Developing   students’   computer   skills,   such   as   word-processing, spreadsheet operations, and email;

x Using ICT for facilitating students' responsibility for their own learning; and

x Using  ICT  to  augment  and  improve  students’  learning. If the   answer   was   “yes”,   the   principals   were   further   asked   to   indicate  

whether the monitoring was done through reviewing lesson plans, teacher self-evaluation, observing classrooms or other means. Table 4.7 presents the percentages of students at schools where   the   principal   monitors   teachers’   ICT   usage  for  students’  learning.  These three items form a reliable scale and is labelled as P_MONSULRN.

97

Tabl

e 4.6

Per

cent

ages

of s

tude

nts a

t sch

ools

whe

re th

e pri

ncip

als e

xpec

t and

requ

ire t

each

ers t

o ha

ve d

iffer

ent t

echn

olog

ical

ped

agog

ical

kn

owle

dge a

nd c

omm

unic

atio

n sk

ills

52(4

.6)

30(5

.1)

23(4

.8)

86(3

.3)

57(5

.9)

26(4

.5)

68(5

.1)

40(5

.6)

30(5

.0)

16(3

.5)

55(1

.6)

33(1

.3)

40(1

.4)

51(1

.5)

47(1

.5)

35(1

.3)

66(1

.5)

42(1

.6)

20(1

.1)

26(1

.2)

42(6

.4)

T36

(5.2

)63

(6.3

)S

82(3

.9)S

60(5

.1)

S34

(4.6

)81

(3.3

)S

36(5

.6)

6(1

.7)T

20(4

.2)

54(1

0.5)

28(5

.8)

37(7

.2)

50(8

.8)

56(1

0.7)

32(6

.9)

55(9

.7)

54(1

0.7)

33(7

.7)

34(6

.7)

*P

erce

ntag

es re

flect

prin

cipa

ls th

at s

elec

ted

"Exp

ecte

d &

requ

ired"

#Th

ese

scal

e sc

ores

for H

K w

ere

cons

truct

ed b

ased

on

conf

irmat

ory

fact

or a

naly

ses

resu

lts o

n th

e H

K d

ata.

( )

Sta

ndar

d er

rors

app

ear i

n pa

rent

hese

s. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

SM

ore

than

10

perc

enta

ge p

oint

s ab

ove

ICIL

S a

vera

geU

Sig

nific

antly

abo

ve IC

ILS

ave

rage

VS

igni

fican

tly b

elow

ICIL

S a

vera

geT

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

belo

w IC

ILS

ave

rage

Educ

atio

nal S

yste

m

Hon

g K

ong

(SAR

)IC

ILS

201

3 av

erag

eAu

stra

liaK

orea

, Rep

. of

Prin

cipal

's ex

pect

ed e

-lear

ning

skill

s & k

now

ledg

e of

teac

hers

that

are

inclu

ded

in sc

ale

scor

e P_

EXPT

PK#

Usin

g IC

T to

de

velo

p au

then

tic (r

eal-

life)

as

sign

men

ts

for s

tude

nts

Inte

grat

ing

ICT

into

teac

hing

an

d le

arni

ng

Usin

g su

bjec

t-sp

ecifi

c le

arni

ng

softw

are

(e.g

. tu

toria

ls,

sim

ulat

ion)

Usin

g e-

portf

olio

s fo

r as

sess

men

t

Inte

grat

ing

Web

-ba

sed

lear

ning

in

thei

rin

stru

ctio

nal

prac

tice

Usin

g IC

T-ba

sed

form

s of

stu

dent

asse

ssm

ent

Usin

g IC

T fo

r m

onito

ring

stud

ent

prog

ress

Com

mun

icat

ing

with

oth

er

staf

f via

ICT

Colla

bora

ting

with

oth

er

teac

hers

via

ICT

Com

mun

icat

ing

with

par

ents

vi

a IC

T

98

First of all, classroom observation is the most common method that principals use to monitor   teachers’   ICT   use   for   two of the aspects in students’   learning:  developing   students’   computer   skills,   and   using   ICT   to   augment   and   improve  students’   learning  with  57%  and  51%  respectively  of   the  participating   students’  principals reported doing so. Similar percentages were also observed in Australia, and Korea.

On   the   other   hand,   the   steps   taken   by  principals   to  monitor   teacher’s   ICT  usage to facilitate   students’   responsibility   for   their   own   learning   were   more  diversified across different countries. Internationally, 47% of students were in schools where this was monitored through classroom observations. Among the three  education  systems  compared  in  this  publication,  66%  of  the  Korean  students’  schools   monitored   teachers’   ICT   usage   for   this   aspect   through classroom observations.  In  contrast,  most  Australian  students’  schools  monitor  this  through  teacher self-evaluation   (55%),  while  most  Hong  Kong  students’   schools  monitor  this through other means (48%). Monitoring   of   teachers’   ICT   use   for   more   advanced/specific areas of learning (P_MONSUADV) A  similar  question  on  whether  and  how  school  principals  monitor  teachers’  usage  of ICT for developing the following more advanced/specific areas of learning:

x Developing   students’   understanding   and   skills   relating to safe and appropriate use of ICT;

x Developing  students’  proficiency  in  accessing  and  using  information  with  ICT; and

x Developing collaborative and organizational skills.

Findings from the above question is presented in Table 4.8. Among the 21 ICILS 2013 participating education systems, classroom observation was the main way that school principals used to monitor these three aspects of teacher ICT use. On the other hand, the most popular means of monitoring adopted in different countries were diverse.

The most popular means of monitoring for all of the above three types of ICT usage was through teacher self-evaluation for Australia (at 55% to 58%), and through classroom observations for Korea (at 44% to 48%). For Hong Kong, the monitoring   of   teachers’   ICT   use   for   developing   students’   collaborative   and  organizational skills was mostly done through classroom observations (43%), while the most popular means of monitoring for the other two types of ICT usages were by other means (46% to 48%). The scale score formed from responses to these three items collected from Hong Kong is labelled P_MONSUADV.

99

23(5

.5)

26(3

.7)

33(4

.6)

46(6

.5)

22(4

.7)

35(4

.4)

45(5

.5)

48(6

.3)

26(4

.8)

27(4

.6)

43(4

.9)

37(4

.9)

26(1

.3)

25(1

.3)

47(1

.7)

38(1

.7)

25(1

.3)

27(1

.3)

52(1

.6)

37(1

.7)

17(1

.2)

25(1

.3)

42(1

.6)

36(1

.6)

16(2

.4)V

55(5

.9)S

46(6

.0)

46(5

.6)

21(3

.0)

58(5

.7)S

53(5

.6)

44(6

.2)

15(2

.3)

58(5

.7)S

44(5

.3)

36(5

.3)

8(3

.0)T

18(4

.9)

44(9

.7)

32(1

2.5)

7(2

.5)T

16(4

.5)T

48(9

.6)

31(1

2.5)

7(2

.4)T

12(3

.4)T

48(8

.3)

32(1

0.2)

#Th

ese

scal

e sc

ores

for H

K we

re c

onst

ruct

ed b

ased

on

conf

irmat

ory

fact

or a

nalys

es re

sults

on

the

HK d

ata.

( )

Sta

ndar

d er

rors

app

ear i

n pa

rent

hese

s. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

SM

ore

than

10

perc

enta

ge p

oint

s ab

ove

ICIL

S av

erag

eU

Sign

ifica

ntly

abov

e IC

ILS

aver

age

VSi

gnifi

cant

ly be

low

ICIL

S av

erag

eT

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

belo

w IC

ILS

aver

age

Hong

Kon

g (S

AR)

Yes,

by

revi

ewin

g le

sson

pl

ans

Yes,

thro

ugh

teac

her s

elf-

eval

uatio

n

Yes,

thro

ugh

obvs

ervi

ng

class

room

s

Educ

atio

nal

Syst

em

ICIL

S 20

13 a

vera

geAu

stra

liaKo

rea,

Rep

. of

Yes,

by o

ther

m

eans

Prin

cipal

's m

onito

ring o

f tea

cher

s' IC

T us

e fo

r dev

elop

ing s

tude

nts'

adva

nced

ICT

skill

s tha

t are

inclu

ded

in sc

ale

scor

e P_

MON

SUAD

V#

Yes,

by

revi

ewin

g le

sson

pla

ns

Yes,

thro

ugh

teac

her s

elf-

eval

uatio

n

Deve

loping

 stud

ents’  u

nderstan

ding

 and

 skills

rela

ting

to sa

fe a

nd a

ppro

pria

te u

se o

f ICT

Deve

loping

 stud

ents’  p

rofic

ienc

y  in  accessin

g  an

d  using  inform

ation  with

 IC

TDe

velo

ping

colla

bora

tive

and

orga

niza

tiona

l ski

lls

Yes,

thro

ugh

obvs

ervi

ng

class

room

s

Yes,

by o

ther

m

eans

Yes,

by

revi

ewin

g le

sson

pl

ans

Yes,

thro

ugh

teac

her s

elf-

eval

uatio

n

Yes,

thro

ugh

obvs

ervi

ng

class

room

s

Yes,

by o

ther

m

eans

Tabl

e 4.8

Percentages  of  students  at  schools  where  the  principals  use  various  means  to  m

onitor  teachers’  ICT  use

36(5

.6)

41(5

.5)

55(4

.5)

38(5

.5)

17(4

.0)

30(4

.0)

36(4

.8)

48(6

.2)

22(4

.8)

39(4

.6)

53(6

.1)

33(4

.7)

42(1

.6)

30(1

.4)

57(1

.6)

23(1

.3)

20(1

.3)

24(1

.2)

47(1

.6)

30(1

.4)

31(1

.4)

29(1

.4)

51(1

.7)

31(1

.7)

36(6

.2)

41(5

.8)

53(5

.6)

32(5

.2)

14(2

.6)V

55(5

.2)S

44(5

.6)

30(5

.2)

32(6

.7)

51(5

.1)S

55(5

.7)

32(5

.2)

10(3

.4)T

12(3

.2)T

64(8

.0)

10(2

.9)T

6(2

.7)T

12(3

.5)T

66(8

.5)S

17(5

.0)

T9

(3.1

)T

14(4

.2)T

54(9

.6)

26(1

1.7)

#Th

ese

scal

e sc

ores

for H

K w

ere

cons

truct

ed b

ased

on

conf

irmat

ory

fact

or a

naly

ses

resu

lts o

n th

e H

K d

ata.

( )

Sta

ndar

d er

rors

app

ear i

n pa

rent

hese

s. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

SM

ore

than

10

perc

enta

ge p

oint

s ab

ove

ICIL

S a

vera

geU

Sig

nific

antly

abo

ve IC

ILS

ave

rage

VS

igni

fican

tly b

elow

ICIL

S a

vera

geT

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

belo

w IC

ILS

ave

rage

Prin

cipal

's m

onito

ring o

f tea

cher

s' IC

T us

e fo

r stu

dent

lear

ning

that

are

inclu

ded

in sc

ale

scor

e P_

MON

SULR

N#

Yes,

by

revi

ewin

g le

sson

pla

ns

Yes,

thro

ugh

teac

her s

elf-

eval

uatio

n

Yes,

thro

ugh

obvs

ervi

ng

class

room

s

Yes,

by o

ther

m

eans

Yes,

by

revi

ewin

g le

sson

pl

ans

Yes,

thro

ugh

teac

her s

elf-

eval

uatio

n

Yes,

thro

ugh

obvs

ervi

ng

class

room

s

Yes,

by o

ther

m

eans

Yes,

by

revi

ewin

g le

sson

pl

ans

Yes,

thro

ugh

teac

her s

elf-

eval

uatio

n

Usin

g  ICT  to  aug

men

t  and

 improv

e  stud

ents’  lea

rning

Usin

g IC

T fo

r fac

ilita

ting

stud

ents

' res

pons

ibili

ty fo

r the

ir ow

n le

arni

ngDe

veloping

 stud

ents’  com

puter  s

kills  

(suc

h as

wor

d-pr

oces

sing,

spre

adsh

eet o

pera

tions

, and

em

ail)

Educ

atio

nal

Syst

emH

ong

Kon

g (S

AR)

ICIL

S 2

013

aver

age

Aust

ralia

Kor

ea, R

ep. o

f

Yes,

thro

ugh

obvs

ervi

ng

class

room

s

Yes,

by o

ther

m

eans

Tabl

e 4. 7

Per

cent

ages

of s

tude

nts a

t sch

ools

where  the  principals  use  various  means  to  m

onitor  teachers’  ICT  use  to  develop  

students’  advanced  IC

T  skills

100

4.2.4 The extent to which principals took main responsibility for ICT management and implementation (P_RESICTM and P_RESICTU)

Principals were asked in the survey to indicate who within their schools took the

main responsibility for different aspects of ICT management connected with ICT

hardware and software management (purchasing/supplying ICT equipment,

selecting software to be used and maintaining ICT equipment), and ICT use aspects

(choosing whether ICT is used in teaching, implementing ICT-based approaches in

teaching and implementing ICT-based approaches in administration). Table 4.9

presents the related descriptive results, which gave rise to two scales, P_RESICTM

and P_RESICTU.

Table 4.9 Percentages of students at schools where the principals took main responsibilities for different aspects of ICT management and implementation

Internationally, the aspect of ICT implementation that had the highest

proportion of principals taking key responsibility for was implementing ICT-based

approaches in administration, with a mean of 81%. Compared to the international

and Australian means, the percentages of Hong Kong principals taking main

responsibility for the various aspects of ICT management were relatively low,

except for the implementation of ICT-based approaches in administration. In

general, principals were least likely to take main responsibility for ICT

maintenance issues.

It is also interesting to note that none of the respective percentages for Korean

principals were higher than 25%, indicating that for Korean schools, most of the

ICT-specific management and implementation responsibilities were devolved to

other staff members.

33 (5.5) 21 (5.8) 5 (2.1) 29 (5.8) 36 (6.2) 80 (4.8)67 (1.5) 41 (1.5) 26 (1.3) 56 (1.6) 45 (1.5) 81 (1.2)51 (5.3) T 28 (5.2) T 20 (4.9) 49 (6.1) 53 (5.3) 89 (3.6)25 (5.7) T 13 (3.3) T 8 (2.5) T 6 (2.1) T 5 (1.7) T 19 (4.5) T

* Percentages reflect principals that selected "Yes taking main responsibility"# These scale scores for HK were constructed based on confirmatory factor analyses results on the HK data. ( ) Standard errors appear in parentheses. Because some results are rounded to the nearest whole number,

some totals may appear inconsistent. S More than 10 percentage points above ICILS averageU Significantly above ICILS averageV Significantly below ICILS averageT More than 10 percentage points below ICILS average

Educational System

Hong Kong (SAR)ICILS 2013 averageAustraliaKorea, Rep. of

Implementing ICT based

approaches in administration

Principal's responsibility for ICT soft and hardware management that are included in

scale score P_RESICTM#

Principal's responsibility for e-Learning curriculum issues that are included in scale

score P_RESICTU#

Purchasing/supplying ICT equipment

Selecting software to be

used

Maintaining ICT equipment

Choosing whether ICT is

used in teaching

Implementing ICTbased

approaches in teaching

101

4.2.5 The extent to which schools had measures regarding ICT access and use (P_ProvStACC)

Principals were asked if they had implemented measures with regard to the

following aspects of ICT access and use:

x Setting up security measures to prevent unauthorised system access or

entry;

x Restricting the number of hours students are allowed to sit at a computer;

x Student access to school computers outside class hours (but during school

hours);

x Student access to school computers outside school hours;

x Honouring of intellectual property rights (e.g. software copyrights);

x Prohibiting access to inappropriate material (e.g. pornography, violence);

x Playing games on school computers;

x Giving the local community (parents and/or others) access to school

computers and/or the Internet; and

x Providing students with their own laptop computers and/or other mobile

learning devices for use at school and at home.

Hong Kong principals were most concerned about ensuring that students

would not access unauthorized or inappropriate sites, and that they would honour

intellectual property rights; all reported having measures to ensure these

conditions. Internationally, these are also the aspects that a vast majority of

students’  principals  reported  having  implemented  relevant  measures.    

Hong Kong principals were least concerned about restricting the total

number of hours students were allowed to sit in front of a computer, with only 33%

of  students’  principals  reported  having  measures  in  place  regarding  this.  An  even  lower  percentage  of  Australian  students’  principals  reported  having  this  type  of  measures in place (18%), whereas the respective percentage for Korea was much

higher, at 64%. Table 4.10 presents the mean percentages of students in schools

with measures in place for the above aspects of ICT access and use. The table also

indicates the three items that contributed to the construction of the scale

P_ProvStACC.

102

Tabl

e 4.1

0 Pe

rcen

tage

s of s

tude

nts a

t sch

ools

whe

re th

e pri

ncip

als t

ook

mai

n re

spon

sibi

litie

s for

diff

eren

t asp

ects

of I

CT

man

agem

ent

87(3

.7)�

72(4

.5)�

43(5

.6)

100

(0.0

)�

33(5

.8)�

100

(0.0

)�

100

(0.0

)�

77(4

.4)�

39(5

.4)�

80(0

.9)

52(1

.1)

35(1

.0)

94(0

.5)

52(1

.1)

89(0

.7)

97(0

.4)

68(1

.0)

47(1

.1)

77(3

.3)

63(3

.6)S

80(3

.2)S

100

(0.0

)U

18(3

.1)T

95(1

.5)U

99(0

.6)U

90(2

.7)S

35(3

.7)T

71(3

.3)V

66(4

.0)S

33(3

.7)

97(1

.4)

64(3

.9)S

95(1

.7)U

96(1

.4)

66(3

.6)

72(3

.7)S

* P

erce

ntag

es re

flect

prin

cipa

ls th

at s

elec

ted

"Yes

"#

Thes

e sc

ale

scor

es fo

r HK

wer

e co

nstru

cted

bas

ed o

n co

nfirm

ator

y fa

ctor

ana

lyse

s re

sults

on

the

HK

dat

a.

( ) S

tand

ard

erro

rs a

ppea

r in

pare

nthe

ses.

Bec

ause

som

e re

sults

are

roun

ded

to th

e ne

ares

t who

le n

umbe

r, so

me

tota

ls m

ay a

ppea

r inc

onsi

sten

t. S

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

abov

e IC

ILS

ave

rage

US

igni

fican

tly a

bove

ICIL

S a

vera

geV

Sig

nific

antly

bel

ow IC

ILS

ave

rage

TM

ore

than

10

perc

enta

ge p

oint

s be

low

ICIL

S a

vera

ge

Educ

atio

nal S

yste

mH

ong

Kon

g (S

AR)

ICIL

S 2

013

aver

age

Aust

ralia

Kor

ea, R

ep. o

f

Play

ing

gam

es

on s

choo

l co

mpu

ters

Giv

ing

the

loca

l co

mm

unity

(p

aren

ts

and/

or o

ther

s)

acce

ss to

sc

hool

co

mpu

ters

an

d/or

the

Inte

rnet

Prov

idin

g st

uden

ts w

ith

thei

r own

la

ptop

co

mpu

ters

an

d/or

oth

er

mob

ile

lear

ning

de

vice

s fo

r us

e at

sch

ool

and

at h

ome

Setti

ng u

p se

curit

y m

easu

res

to

prev

ent

unau

thor

ised

sy

stem

ac

cess

or

entry

Rest

rictin

g th

e nu

mbe

r of

hour

s st

uden

ts a

re

allo

wed

to s

it at

a c

ompu

ter

Stud

ent

acce

ss to

sc

hool

co

mpu

ters

ou

tsid

e cl

ass

hour

s (b

ut

durin

g sc

hool

ho

urs)

Stud

ent

acce

ss to

sc

hool

co

mpu

ters

ou

tsid

e sc

hool

hou

rs

Hono

urin

g of

in

telle

ctua

l pr

oper

ty

right

s (e

.g.

softw

are

copy

right

s)

Proh

ibiti

ng

acce

ss to

in

appr

opria

te

mat

eria

l (e.

g.

porn

ogra

phy,

vi

olen

ce)

Scho

ol p

rovi

de s

tude

nts'

acc

ess

to

com

pute

rs a

t sch

ool t

hat a

re in

clud

ed

in s

cale

sco

re P

_Pro

vStA

CC#

103

4.2.6 The extent to which teachers participated in different forms of professional development as reported by principals (P_TPDpart)

Making provisions for teacher learning regarding ICT use is one major strategic area in any ICT in education policy. These could be organized as formal courses taught by experts, as school-based peer taught events, or as practice oriented events such as peer class observations, or meetings and informal discussions. Principals were asked to indicate how many teachers in their schools participated in the following forms of professional development about ICT for teaching and learning using a four point Likert scale (none or almost none, some, many, all or almost all):

x Participating in courses on the use of ICT in teaching provided by the school;

x Working with another teacher who has attended a course and then trains other teachers;

x Discussing the use of ICT in education as a regular item during meetings of the teaching staff;

x Observing colleagues using ICT in their teaching; x Discussing within groups of teachers about using ICT in their teaching; x Participating in a [community of practice] concerned with ICT in teaching; x Participating in courses conducted by an external agency or expert; and x Participating in professional learning programs delivered through ICT.

Table 4.11 shows the percentage of students whose principals responded that

either  “many”  or  “all  or  almost  all”  of  their  schools’  teachers  have  participated  in  each kind of professional development activities. All these items were included in the construction of the scale P_TPDpart.

Internationally, the most popular form of ICT-related professional development activity was courses provided by the school, followed by informal discussions within groups of teachers as well as discussions on ICT use as a regular item embedded into staff meetings. Australian principals reported high participation in all forms of professional development, particularly the three forms of professional development that are also internationally most popular. In both Hong Kong and Korea, the most popular professional development activities were courses provided by the school and observing colleagues using ICT in their teaching.

However, the levels of participation reported by Hong Kong principals were very much lower than even the international average. This may be one of the reasons for the low levels of ICT adoption in teaching and student learning reported by teachers in Hong Kong.

104

Tabl

e 4.1

1 Pe

rcen

tage

s of s

tude

nts a

t sch

ools

whe

re te

ache

rs p

artic

ipat

e in

diffe

rent

ICT-

rela

ted

prof

essi

onal

dev

elop

men

t as r

epor

ted

by

thei

r pri

ncip

als

39(5

.6)

15(4

.4)

18(4

.5)

36(5

.3)

19(4

.4)

11(3

.9)

21(4

.6)

26(5

.2)

68(1

.0)

47(1

.1)

53(1

.1)

44(1

.0)

56(1

.1)

29(1

.0)

39(1

.0)

39(1

.0)

80(2

.6)S

67(3

.6)S

75(2

.8)S

55(3

.6)S

72(3

.2)S

44(3

.6)S

41(3

.4)

58(3

.0)S

61(4

.3)

48(4

.2)

36(3

.7)T

60(4

.1)S

37(4

.4)T

25(3

.8)

34(4

.1)

45(4

.0)

* P

erce

ntag

es re

flect

prin

cipa

ls th

at s

elec

ted

"All

or a

lmos

t all"

or "

Man

y".

#Th

ese

scal

e sc

ores

for H

K w

ere

cons

truct

ed b

ased

on

conf

irmat

ory

fact

or a

naly

ses

resu

lts o

n th

e H

K d

ata.

( )

Sta

ndar

d er

rors

app

ear i

n pa

rent

hese

s. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

SM

ore

than

10

perc

enta

ge p

oint

s ab

ove

ICIL

S a

vera

geU

Sig

nific

antly

abo

ve IC

ILS

ave

rage

VS

igni

fican

tly b

elow

ICIL

S a

vera

geT

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

belo

w IC

ILS

ave

rage

Educ

atio

nal S

yste

mH

ong

Kon

g (S

AR)

ICIL

S 2

013

aver

age

Aust

ralia

Kor

ea, R

ep. o

f

Teac

hers’  p

artic

ipation  in  profess

iona

l  dev

elop

men

t  as  repo

rted  by

 the  principa

l  tha

t  are  in

clud

ed  in

 sca

le  

scor

e P_

TPDp

art#

Parti

cipa

ting

in c

ours

es

cond

ucte

d by

an

ext

erna

l ag

ency

or

expe

rt

Parti

cipa

ting

in

prof

essi

onal

le

arni

ng

prog

ram

s de

liver

ed

thro

ugh

ICT

Parti

cipa

ting

in c

ours

es o

n th

e us

e of

ICT

in te

achi

ng

prov

ided

by

the

scho

ol

Wor

king

with

an

othe

r te

ache

r who

ha

s at

tend

ed

a co

urse

and

th

en tr

ains

ot

her t

each

ers

Disc

ussi

ng

the

use

of IC

T in

edu

catio

n as

a re

gula

r ite

m d

urin

g m

eetin

gs o

f th

e te

achi

ng

staf

f

Obs

ervi

ng

colle

ague

s us

ing

ICT

in

thei

r tea

chin

g

Disc

ussi

ng

with

in g

roup

s of

teac

hers

ab

out u

sing

IC

T in

thei

r te

achi

ng

Parti

cipa

ting

in a

[c

omm

unity

of

prac

tice]

co

ncer

ned

with

ICT

in

teac

hing

105

4.2.7 Principal’s  priorities  for  facilitating  use  of  ICT (P_PriAcComp,

P_PriELRes and P_PriPedaUse)

Principals play a major role in many important decisions regarding resource

allocation and staffing that affect ICT use in schools. Principals were asked in the

questionnaire to indicate the priorities they gave to the following to facilitate the

use of ICT in teaching and learning in their schools through a 4-point Likert scale

(not a priority, low priority, medium priority, high priority):

x Increasing the numbers of computers per student in the school;

x Increasing the number of computers connected to the Internet;

x Increasing the bandwidth of Internet access for the computers connected

to the Internet;

x Increasing the range of digital learning resources;

x Establishing or enhancing an online learning support platform;

x Providing for participation in professional development on pedagogical

use of ICT;

x Increasing the availability of qualified technical personnel to support the

use of ICT;

x Providing teachers with incentives to integrate ICT use in their teaching;

x Providing more time for teachers to prepare lessons in which ICT is used;

and

x Increasing the professional learning resources for teachers in the use of

ICT.

Table 4.12 presents the percentages of principals who indicated medium or

high priority given to the ways of facilitation listed above. From the table, it is clear

that  internationally,  a  large  majority  of  surveyed  students’  principals  found  all  of  the measures listed above to be at least of medium priority (varying between 78%

and 93%). Furthermore, the profile of priorities indicate that digital learning

resources, providing for participation in professional development on pedagogical use of ICT and increasing the professional learning resources for teachers in the use of ICT were

considered   to   be   of   priority   by   a   greater   percentage   of   students’   principals

compared to computer access and Internet connectivity.

On  the  other  hand,  the  priorities  given  by  Hong  Kong  students’  principals  were lower than the international average, except for establishing or enhancing an online learning support platform, which was 87% as compared to 79% for the

international average. In fact, establishing or enhancing online learning support platform also given high priorities by principals of Australian and Korea students,

at 90% and 94% respectively. In addition, the priorities given by Hong Kong

students’   principals   to   supporting   professional   learning   of   teachers  were   lower  than that given to increasing the bandwidth of Internet connections in the school.

106

50(4

.3)

74

(5.1

)

84(3

.9)

83

(4.3

)

87(3

.3)

79

(3.7

)

68(5

.0)

69

(4.8

)

55(6

.0)

80

(4.1

)88

(0.7

)89

(0.7

)89

(0.7

)93

(0.6

)79

(0.9

)91

(0.6

)84

(0.9

)86

(0.7

)78

(0.9

)91

(0.6

)81

(2.8

)V

79(3

.0)T

84(2

.6)V

93(2

.0)

90(2

.1)S

97(1

.2)U

80(2

.8)

68(3

.6)T

54(4

.0)T

90(2

.2)

65(3

.6)T

71(3

.5)T

75(3

.8)T

89(2

.7)

94(2

.0)S

89(2

.5)

84(3

.1)

90(2

.6)

87(2

.7)U

96(1

.7)U

*P

erce

ntag

es re

flect

prin

cipa

ls th

at s

elec

ted

"Hig

h pr

iorit

y" o

r "M

ediu

m p

riorit

y".

#Th

ese

scal

e sc

ores

for H

K w

ere

cons

truct

ed b

ased

on

conf

irmat

ory

fact

or a

naly

ses

resu

lts o

n th

e H

K d

ata.

( )

Sta

ndar

d er

rors

app

ear i

n pa

rent

hese

s. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

mor

e th

an 3

sco

re p

oint

s ab

ove

ICIL

S a

vera

geS

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

abov

e IC

ILS

ave

rage

US

igni

fican

tly a

bove

ICIL

S a

vera

geV

Sig

nific

antly

bel

ow IC

ILS

ave

rage

mor

e th

an 3

sco

re p

oint

s be

low

ICIL

S a

vera

geT

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

belo

w IC

ILS

ave

rage

Prio

ritie

s giv

en to

impr

ove

acce

ss to

com

pute

rs a

nd

inte

rnet

in sc

hool

that

are

incl

uded

in sc

ale

scor

e P_

PriA

cCom

p#

Prio

ritie

s giv

en to

incr

ease

the

rang

e of

e-L

earn

ing

reso

urce

s in

scho

ol th

at a

re in

clud

ed in

scal

e sc

ore

P_Pr

iELR

es#

Prio

ritie

s giv

en to

impr

ove

supp

orts

for t

each

ers o

n pe

dago

gica

l use

of

ICT

in sc

hool

that

are

incl

uded

in sc

ale

scor

e P_

PriP

edaU

se#

Prov

idin

g fo

r pa

rtic

ipat

ion

in

prof

essi

onal

de

velo

pmen

t on

pe

dago

gica

l us

e of

ICT

Incr

easi

ng th

e pr

ofes

sion

al

lear

ning

re

sour

ces

for

teac

hers

in

the

use

of IC

T

Incr

easi

ng th

e av

aila

bilit

y of

qu

alifi

ed

tech

nica

l pe

rson

nel t

o su

ppor

t the

us

e of

ICT

Prov

idin

g te

ache

rs w

ith

ince

ntiv

es to

in

tegr

ate

ICT

use

in th

eir

teac

hing

Prov

idin

g m

ore

time

for

teac

hers

to

prep

are

less

ons

in

whi

ch IC

T is

us

ed

Incr

easi

ng th

e nu

mbe

rs o

f co

mpu

ters

pe

r stu

dent

in

the

scho

ol

Incr

easi

ng th

e nu

mbe

r of

com

pute

rs

conn

ecte

d to

th

e In

tern

et

Incr

easi

ng th

e ba

ndw

idth

of

Inte

rnet

ac

cess

for t

he

com

pute

rs

conn

ecte

d to

th

e In

tern

et

Aust

ralia

Kor

ea, R

ep. o

f

Incr

easi

ng th

e ra

nge

of

digi

tal l

earn

ing

reso

urce

s

Esta

blis

hing

or

enh

anci

ng

an o

nlin

e le

arni

ng

supp

ort

plat

form

Educ

atio

nal S

yste

m

Hon

g K

ong

(SAR

)IC

ILS

201

3 av

erag

e

Tabl

e 4.1

2 Pe

rcen

tage

s of s

tude

nts a

t sch

ools

whe

re p

rinc

ipal

s ind

icat

e med

ium

or h

igh

prio

rity

to w

ays o

f fac

ilita

ting

use o

f IC

T in

te

achi

ng a

nd le

arni

ng

107

In   contrast,   Australian   and   Korean   principals’   top   priorities   reflect   their  concern  about  teacher’s  pedagogical  use  of  ICT  as  well  as  the  range  of  e-Learning

resources at school. Providing for participation in professional development on pedagogical use of ICT was  considered  by  97%  of  Australian  students’  principals  and  89%  of  Korea  students’  principals  as  of  at  least  medium  priority.  Among  Korean  students’  principals,  the  top  three  priorities  were:  increasing the professional learning resources for teachers in the use of ICT (96%), establishing or enhancing online learning support platforms (94%), and providing teachers with incentives to integrate ICT use in their teaching (90%). Interestingly, only 68-69% of Australian and Hong Kong

students’  principals  indicated providing teachers with incentives to integrate ICT

use in their teaching as of medium or high priority.

Three  scale  scores  on  principals’  priorities  were  constructed  from  responses  to the items in this question as presented in Table 4.12: improving computer and

Internet access (P_PriAcComp), increasing the range of learning resources

(P_PriELRes), and improving support for teachers on pedagogical use of ICT

(P_PriPedaUse) in the school.

4.2.8 Obstacles  that  hinder  school’s  capacity  to  realize  its e-Learning goals (NP_ObsInsufH&S, NP_ObsbyTs, NP_ObsCurr, NP_InsufRes)

The Hong Kong version of ICILS principal questionnaire contains one national

item that invited principals to rate whether the issues listed below pose hindrance

(Not at all, a little, somewhat,  a  lot  or  not  applicable)  to  their  school’s  capacity  to  realize its e-Learning goals:

x Insufficient qualified technical personnel to support the use of ICT;

x Insufficient number of computers connected to the Internet;

x Insufficient Internet bandwidth or speed;

x Insufficient ICT equipment for student use in learning;

x Computers are out of date;

x Teachers' lack of ICT skills;

x Insufficient time for teachers to implement e-learning;

x Teachers do not have sufficient knowledge to implement new pedagogical

approaches for e-learning;

x Pressure to score highly on standardised tests;

x Prescribed curricula are too strict;

x Insufficient or inappropriate space to accommodate the school's

pedagogical approaches;

x Insufficient budget for the needs of ICT implementation (e.g. LMS); and

x Using ICT for teaching and/or learning is not a goal of our school.

108

Table  4.13  reports  the  percentages  of  principals  who  indicate  “somewhat”  or  “a   lot”  hindrance   to   the   issues   listed.  The   top  issues   that  Hong  Kong  principals  indicated as hindrances to their school were pedagogy related, for example:

insufficient time for teachers to implement e-Learning (59%) is the number one

hindrance as perceived by Hong Kong principals, followed by insufficient budget for the needs of ICT implementation (e.g. LMS) (46%) as well as insufficient qualified technical personnel to support the use of ICT (35%). More than one third (34%) of Hong

Kong principals indicated that pressure to score highly on standardized tests was

an obstacle.

The data further suggest that lack of hardware or general ICT skills among

teachers were not big obstacles to Hong Kong principals. Only 13%-19% of Hong

Kong   students’   principals   indicated   insufficient   computers   (13%),   Internet  bandwidth (14%), and ICT equipment (19%) were obstacles to their school.

Teacher’s   lack   of   ICT   skills   was   also   not   a   major   concern   among   Hong   Kong  principals, as only 11% of principals included this as an obstacle in their responses.

Four scales related to the nature of the obstacles to implementing e-learning

as   perceived   by   principals’   were   constructed:   insufficient   hard- and soft- ware

(NP_ObsInsufH&S), teacher-related obstacles (NP_ObsbyTs), curriculum and

assessment related obstacles (NP_ObsCurr) and inadequate budget and/or other

resources (NP_InsufRes). Their respective compositions are presented in Table 4.13.

4.3  Teacher’s  ICT-using Pedagogy

Teachers play the biggest role in determining the learning experiences of students.

The role that ICT plays in the learning lives of students could play an important

role  in  influencing  students’  CIL  outcomes. In the ICILS 2013 teacher questionnaire, we asked teachers to report on what

they  use  and  how  they  use  ICT  in  their  teaching  and  in  their  students’  learning.  In  addition, we tried to find out about their confidence in various general purpose

and pedagogical uses of ICT, as well as the extent to which they put emphasis on

supporting   their   students’   development   of   various   CIL   capabilities.   The  descriptive results from the teacher questionnaire are reported in this section to

provide  a  comprehensive  picture  of  teachers’  ICT-using pedagogy in Hong Kong,

and in comparison with their international, Australian and Korean counterparts.

Similar to the reporting on scale scores constructed from the relevant

questionnaire responses, we also identify the scales constructed from the teacher

questionnaire responses in the sections where the relevant related descriptive

statistics.

109

Tabl

e 4.1

3 Pe

rcen

tage

s of H

ong

Kon

g principals  who  indicate  “somew

hat”  or  “a  lot”  of  hindran

ce  cau

se  by  issues  listed  to  their  school’s  

capa

city

to re

aliz

e e-L

earn

ing

capa

city

35(4

.4)

13(3

.3)

14(3

.2)

19(4

.2)

28(4

.8)

11(2

.9)

59(5

.5)

26(4

.3)

34(5

.5)

22(4

.6)

31(4

.6)

46(5

.8)

9(3

.1)

*P

erce

ntag

es re

flect

prin

cipa

ls th

at s

elec

ted

"A lo

t".#

Thes

e sc

ale

scor

es fo

r HK

wer

e co

nstru

cted

bas

ed o

n co

nfirm

ator

y fa

ctor

ana

lyse

s re

sults

on

the

HK

dat

a.

( ) S

tand

ard

erro

rs a

ppea

r in

pare

nthe

ses.

Bec

ause

som

e re

sults

are

roun

ded

to th

e ne

ares

t who

le n

umbe

r, so

me

tota

ls m

ay a

ppea

r inc

onsi

sten

t.

Hon

g K

ong

(SAR

)

Insu

fficien

t  ICT

 hard  an

d  so

ftware  hind

er  th

e  sc

hool’s  cap

acity

 to  

real

ize

its e

-Lea

rnin

g go

als

that

are

incl

uded

in s

cale

sco

re

NP_O

bsIn

sufH

&S#

Insu

ffici

ent

qual

ified

te

chni

cal

pers

onne

l to

supp

ort t

he

use

of IC

T

Insu

ffici

ent

num

ber o

f co

mpu

ters

co

nnec

ted

to th

e In

tern

et

Insu

ffici

ent

Inte

rnet

ba

ndw

idth

or

spe

ed

Insu

ffici

ent

ICT

equi

pmen

t fo

r stu

dent

us

e in

le

arni

ng

Com

pute

rs

are

out o

f da

teEd

ucat

iona

l Sy

stem

Insu

ffici

ent b

udge

t, sp

ace,

or

mismatch

 of  g

oals  hinde

r  the

 sch

ool’s

 ca

paci

ty to

real

ize

its e

-Lea

rnin

g go

als

that

are

incl

uded

in s

cale

sco

re

NP_I

nsuf

Res#

Insu

ffici

ent

or

inap

prop

riat

e sp

ace

to

acco

mm

oda

te th

e sc

hool

's

peda

gogi

cal

appr

oach

es

Insu

ffici

ent

budg

et fo

r th

e ne

eds

of

ICT

impl

emen

tat

ion

(e.g

. LM

S)

Usin

g IC

T fo

r tea

chin

g an

d/or

le

arni

ng is

no

t a g

oal o

f ou

r sch

ool

Teac

hers

' la

ck o

f ICT

sk

ills

Insu

ffici

ent

time

for

teac

hers

to

impl

emen

t e-

lear

ning

Teac

hers

do

not h

ave

suffi

cien

t kn

owle

dge

to

impl

emen

t ne

w

peda

gogi

cal

appr

oach

es

for e

-le

arni

ng

Obs

tacl

es fa

ced

by te

ache

rs h

inde

r the

sc

hool’s  cap

a-­city  to

 realize  its

 e-­

Lear

ning

goa

ls th

at a

re in

clud

ed in

sc

ale

scor

e NP

_Obs

byTs

#

Pres

crib

ed

curr

icul

a ar

e to

o st

rict

Pres

sure

to

scor

e hi

ghly

on

st

anda

rdis

ed

test

s

Curr

icul

um a

nd

asse

ssm

ent r

elat

ed

obst

acle

s hi

nder

the

scho

ol’s  cap

acity

 to  

real

ize

its e

-Lea

rnin

g go

als

that

are

incl

uded

in

scal

e sc

ore

NP_O

bsCu

rr#

110

4.3.1 Teacher confidence in using ICT (T_EFGen, T_EFAdv, T_EFPeda)

The ICILS teacher questionnaire invited teachers to indicate their confidence in their own ability to carry out a number of tasks on a computer by themselves using three points Likert scale, (I know how to do this, I could work out how to do this, I do not think I could do this). These tasks ranged from general computer operations to pedagogical uses of ICT:

x Producing a letter using a word-processing program; x E-mailing a file as an attachment; x Storing your digital photos on a computer; x Filing digital documents in folders and sub-folders; x Producing presentations (e.g. [Microsoft PowerPoint®] or a similar

program), with simple animation functions; x Finding useful teaching resources on the Internet; x Using a spreadsheet program (e.g. [Lotus 1 2 3 ®, Microsoft Excel ®]) for

keeping records or analysing data; x Contributing to a discussion forum/user group on the Internet (eg. a wiki

or blog); x Using the Internet for online purchases and payments; x Collaborating with others using shared resources such as [Google Docs®] x Installing software; x Monitoring students' progress; x Preparing lessons that involve the use of ICT by students; and x Assessing student learning.

Table  4.14  reports  the  percentages  of  teachers  who  have  indicated  “I  know  

how  to  do”  the  listed  tasks  in  the  survey.  The  results  indicate  that a large majority of Hong Kong teachers surveyed knew how to perform general and some advanced ICT tasks, with percentages higher than the international average. This stands  in  stark  contrasts  to  the  teachers’  self-reported competence in ICT use for pedagogically   related   tasks,   namely   monitoring   students’   progress   (52%),   and  assessing   students’   learning   (58%),   which   were   much   lower   than   the  corresponding percentages reported by Australian (respectively 86% and 83%) and Korean (respectively 62% and 82%) teachers. Three scales were derived from confirmatory  analysis  on  responses  to  this  question  related  to  teachers’  ICT-related self-efficacy: in general ICT tasks (T_EFGen), in advanced ICT skills (T_EFAdv), and in pedagogical use of ICT (T_EFPeda), as indicated in Table 4.14.

111

94(1

.1)

97

(0.6

)

93(1

.0)

92

(0.9

)

92(0

.8)

94

(0.6

)

74(1

.5)

66

(1.6

)

80(1

.2)

45

(1.5

)

69(1

.5)

52(1

.5)

74

(1.2

)

58(1

.4)

89

(0.3

)89

(0.3

)82

(0.3

)84

(0.3

)76

(0.4

)92

(0.3

)59

(0.4

)58

(0.5

)77

(0.4

)44

(0.5

)47

(0.4

)65

(0.5

)73

(0.4

)71

(0.5

)98

(0.3

)U

98(0

.3)U

93(0

.5)S

94(0

.6)U

87(0

.6)S

96(0

.5)U

74(1

.2)S

60(1

.1)

95(0

.5)S

48(1

.8)U

69(1

.1)S

86(0

.8)S

90(0

.7)S

83(0

.9)S

95(0

.8)U

97(0

.9)U

96(0

.9)S

94(0

.7)U

68(2

.0)V

95(1

.8)

69(1

.1)U

66(1

.5)U

94(0

.8)S

35(1

.1)V

66(1

.8)S

62(1

.7)

84(1

.2)S

82(2

.0)S

*P

erce

ntag

es re

flect

teac

her w

ho h

ave

sele

cted

"I k

now

how

to d

o th

is".

#Th

ese

scal

e sc

ores

for H

K w

ere

cons

truct

ed b

ased

on

conf

irmat

ory

fact

or a

naly

ses

resu

lts o

n th

e H

K d

ata.

( )

Sta

ndar

d er

rors

app

ear i

n pa

rent

hese

s. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

SM

ore

than

10

perc

enta

ge p

oint

s ab

ove

ICIL

S a

vera

geU

Sig

nific

antly

abo

ve IC

ILS

ave

rage

VS

igni

fican

tly b

elow

ICIL

S a

vera

geT

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

belo

w IC

ILS

ave

rage

Educ

atio

nal S

yste

mH

ong

Kon

g (S

AR)

ICIL

S 2

013

aver

age

Aust

ralia

Kor

ea, R

ep. o

f

Mon

itorin

g st

uden

ts'

prog

ress

Cont

ribut

ing

to a

di

scus

sion

fo

rum

/use

r gr

oup

on th

e In

tern

et (e

g. a

wi

ki o

r blo

g)

Teac

her’s

 self-­e

ffica

cy  in

 ped

agog

ical  

use

of IC

T th

at a

re in

clud

ed in

sca

le

scor

e T_

EFPe

da#

Inst

allin

g so

ftwar

e

Asse

ssin

g st

uden

t le

arni

ng

Teac

her’s

 self-­e

ffica

cy  in

 gen

eral  IC

T  task

s  th

at a

re in

clud

ed in

sca

le s

core

T_E

FGen

#

Prod

ucin

g a

lette

r usi

ng a

wo

rd-

proc

essi

ng

prog

ram

E-m

ailin

g a

file

as a

n at

tach

men

t

Stor

ing

your

di

gita

l pho

tos

on a

com

pute

r

Filin

g di

gita

l do

cum

ents

in

fold

ers

and

sub-

fold

ers

Usin

g a

spre

adsh

eet

prog

ram

(e.g

. [L

otus

1 2

3 ®

, M

icro

soft

Exce

l ®])

for

keep

ing

reco

rds

or

anal

ysin

g da

ta

Find

ing

usef

ul

teac

hing

re

sour

ces

on

the

Inte

rnet

Colla

bora

ting

with

oth

ers

usin

g sh

ared

re

sour

ces

such

as

[Goo

gle

Docs

®]

Teac

her’s

 self-­e

ffica

cy  in

 adv

ance

d  ICT  sk

ills  

that

are

incl

uded

in s

cale

sco

re T

_EFA

dv#

Prod

ucin

g pr

esen

tatio

ns

(e.g

. [M

icro

soft

Powe

rPoi

nt®]

or

a s

imila

r pr

ogra

m),

with

si

mpl

e an

imat

ion

func

tions

Usin

g th

e In

tern

et fo

r on

line

purc

hase

s an

d pa

ymen

ts

Prep

arin

g le

sson

s th

at

invo

lve

the

use

of IC

T by

st

uden

ts

Tabl

e 4.1

4 Pe

rcen

tage

s of t

each

ers e

xpre

ssin

g co

nfid

ence

in d

oing

diff

eren

t com

pute

r tas

ks

112

4.3.2  Teachers’  reported  use  of  ICT  tools  in  teaching (T_USEPedaT)

The survey asked teachers to indicate the frequency of their use of the following

ICT tools in teaching using a 4-point Likert scale (never, in some lessons, in most

lessons, in every or almost every lesson):

x Tutorial software or [practice programs];

x Digital learning games;

x Spreadsheets (e.g. [Microsoft Excel®]);

x Interactive digital learning resources (e.g. learning objects)

x e-portfolios;

x Word-processors or presentation software (e.g. [Microsoft Word ®],

[Microsoft PowerPoint ®]);

x Multimedia production tools (e.g. media capture and editing, web

production);

x Concept mapping software (e.g. [Inspiration ®], [Webspiration ®]);

x Data logging and monitoring tools ;

x Simulations and modelling software;

x Social media (e.g. Facebook, Twitter);

x Communication software (e.g. email, blogs);

x Computer-based information resources (e.g. encyclopaedia, wikis,

websites); and

x Graphing or drawing software.

Table 4.15 reports on the percentages of teachers indicating frequent use of

ICT tools (that is, in most lessons, or in every or almost every lesson). Results show

that the highest reported frequent usage is with the use word-processors or

presentation software internationally (30%) as well as in Hong Kong (52%),

Australia (41%) and Korea (47%). Tutorial software, interactive digital resources,

and computer-based  information  resources  had  somewhat  “medium”  usage,  being  reported for frequent usage by more than 10% of by Hong Kong teachers. Most of

the other ICT tools surveyed, such as e-portfolios and concept mapping software,

require deeper pedagogical integration for their use and were reported as

frequently used by single digit percentages of teachers in Hong Kong and

elsewhere. A scale related to teachers’ reported use of pedagogical ICT tools,

T_USEPedaT, was constructed from five of the items in this question, as indicated

in Table 4.15.

113

22(1

.2)

3

(0.6

)

9(1

.0)

13

(1.1

)

2(0

.4)

52(1

.9)

11

(1.0

)

3(0

.6)

3

(0.6

)

3(0

.5)

3

(0.6

)

9(1

.1)

13

(1.0

)

6(0

.7)

15

(0.4

)5

(0.2

)7

(0.3

)15

(0.4

)4

(0.2

)30

(0.4

)8

(0.3

)4

(0.2

)6

(0.2

)3

(0.1

)4

(0.2

)10

(0.3

)23

(0.4

)7

(0.3

)7

(0.6

)V

6(0

.6)

5(0

.5)V

15(0

.8)

3(0

.4)V

41(1

.2)S

10(0

.6)

2(0

.3)V

5(0

.5)

4(0

.5)

1(0

.3)V

15(1

.4)U

31(1

.1)U

5(0

.5)V

28(1

.9)S

7(1

.0)

10(0

.8)U

11(0

.6)V

6(0

.9)

47(1

.9)S

17(2

.0)U

3(0

.7)

5(0

.9)

6(0

.7)U

5(0

.8)U

12(1

.2)

20(1

.0)V

20(2

.4)S

*P

erce

ntag

es re

flect

teac

her w

ho h

ave

sele

cted

"In

mos

t les

sons

" and

"In

ever

y or

alm

ost e

very

less

on".

#Th

ese

scal

e sc

ores

for H

K w

ere

cons

truct

ed b

ased

on

conf

irmat

ory

fact

or a

naly

ses

resu

lts o

n th

e H

K d

ata.

( )

Sta

ndar

d er

rors

app

ear i

n pa

rent

hese

s. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

SM

ore

than

10

perc

enta

ge p

oint

s ab

ove

ICIL

S a

vera

geU

Sig

nific

antly

abo

ve IC

ILS

ave

rage

VS

igni

fican

tly b

elow

ICIL

S a

vera

geT

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

belo

w IC

ILS

ave

rage

Kor

ea, R

ep. o

f

Teac

her’s

 repo

rted  us

e  of  ped

agog

ical  IC

T  tools  

that

are

incl

uded

in s

cale

sco

re T

_USE

Peda

T#

Educ

atio

nal S

yste

m

Inte

ract

ive

digi

tal

lear

ning

re

sour

ces

(e.g

. le

arni

ng

obje

cts)

Hon

g K

ong

(SAR

)IC

ILS

201

3 av

erag

eAu

stra

lia

Conc

ept

map

ping

so

ftwar

e (e

.g.

[Insp

iratio

n ®

], [W

ebsp

irati

on ®

])

Data

lo

ggin

g an

d m

onito

ring

tool

s

Tuto

rial

softw

are

or

[pra

ctic

e pr

ogra

ms]

Digi

tal

lear

ning

ga

mes

Wor

d-pr

oces

sors

or

pr

esen

tatio

n so

ftwar

e (e

.g.

[Mic

roso

ft W

ord

®],

[Mic

roso

ft Po

wer

Poin

t ®

])

Spre

adsh

eet

s (e

.g.

[Mic

roso

ft Ex

cel®

])

Mul

timed

ia

prod

uctio

n to

ols

(e.g

. m

edia

ca

ptur

e an

d ed

iting

, web

pr

oduc

tion)

Com

pute

r-ba

sed

info

rmat

ion

reso

urce

s (e

.g.

web

site

s,

wik

is,

ency

clop

aed

ia)

Gra

phin

g or

dr

awin

g so

ftwar

ee-

portf

olio

s

Sim

ulat

ions

an

d m

odel

ling

softw

are

Soci

al

med

ia (e

.g.

Face

book

, Tw

itter

)

Com

mun

icat

ion

softw

are

(e.g

. em

ail,

blog

s)

Tabl

e 4.1

5 Pe

rcen

tage

s of t

each

ers u

sing

ICT

tool

s in

mos

t, al

mos

t eve

ry, a

nd ev

ery

less

ons

114

4.3.3  Teachers’  reported student use of ICT in different learning tasks (T_StUInno, T_StUTrad, T_StUExt)

The survey also invited teachers to report on frequencies of  students’  usage  of  ICT  (never, sometimes, often) in different types of learning activities, as listed below:

x Working on extended projects (i.e. over several weeks) x Undertaking open-ended investigations or field work x Reflecting on their learning experiences (e.g. by using a learning log) x Communicating with students in other schools on projects x Planning a sequence of learning activities for themselves x Processing and analysing data x Evaluating information resulting from a search x Working on short assignments (i.e. within one week) x Explaining and discussing ideas with other students x Submitting completed work for assessment x Working individually on learning materials at their own pace x Seeking information from experts outside the school x Searching for information on a topic using outside resources

Table 4.16 reports the percentages of teachers who reported that their grade

8 students often use ICT for learning activities in the classroom. The results for most of the learning activities surveyed were very low, at only a single digit percentage. The highest percentages recorded were for working on extended projects (i.e. over several weeks) (12%), and searching for information on a topic using   outside   resources   (11%),  Hong  Kong   students’ ICT usages in all types of learning activities were either equal to, or lower than the corresponding international averages.

The difference in percentage between the Hong Kong and international means  was   largest   for   students’  use  of   ICT   to   search   for   information  on  a   topic  using outside resources, at 18%. The percentages reported by Korean teachers were also lower than the international average for most of the ICT-using   students’  learning activities surveyed, though these averages were still higher than the corresponding one for Hong Kong, except for one activity, which is to use ICT to work on extended projects. This figure was 12% for Hong Kong and 9% for Korea. On the other hand, the corresponding percentages reported by Australian teachers were generally higher than the international mean, with often use of ICT reported by more than 30% in three activities: working on extended projects, submitting completed work for assessment and searching for information on a topic using outside resources. Three scales were constructed from responses to this question on  students’  ICT  use:  in  innovative  student-centred learning tasks (T_StUInno), in traditional learning tasks (T_StUTrad), and in accessing external resources and experts (T_StUExt), as presented in Table 4.16.

115

12(1

.1)

3

(0.7

)

2(0

.5)

2

(0.6

)

2(0

.6)

5

(0.8

)

4(0

.8)

5(0

.7)

5

(0.7

)

7(0

.8)

5

(0.6

)

2(0

.5)

11

(1.2

)

12(0

.3)

8(0

.3)

6(0

.3)

3(0

.2)

11(0

.3)

11(0

.4)

14(0

.4)

20(0

.4)

12(0

.3)

18(0

.4)

16(0

.3)

7(0

.3)

29(0

.5)

31(1

.3)S

16(1

.0)U

6(0

.6)

4(0

.5)

3(0

.4)V

7(0

.7)V

15(0

.9)

31(1

.5)S

15(1

.0)U

32(1

.3)S

28(1

.2)S

4(0

.4)V

32(1

.4)U

9(1

.3)V

5(0

.7)V

4(0

.6)V

4(0

.7)

5(0

.8)V

10(1

.4)

7(1

.0)V

13(1

.4)V

8(0

.9)V

11(0

.9)V

11(1

.2)V

15(1

.7)U

19(2

.1)V

*Pe

rcen

tage

s re

flect

teac

her w

ho h

ave

indi

cate

d "O

ften"

.#

Thes

e sc

ale

scor

es fo

r HK

were

con

stru

cted

bas

ed o

n co

nfirm

ator

y fa

ctor

ana

lyses

resu

lts o

n th

e HK

dat

a.

( ) S

tand

ard

erro

rs a

ppea

r in

pare

nthe

ses.

Bec

ause

som

e re

sults

are

roun

ded

to th

e ne

ares

t who

le n

umbe

r, so

me

tota

ls m

ay a

ppea

r inc

onsi

sten

t. S

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

abov

e IC

ILS

aver

age

USi

gnifi

cant

ly ab

ove

ICIL

S av

erag

eV

Sign

ifica

ntly

belo

w IC

ILS

aver

age

TM

ore

than

10

perc

enta

ge p

oint

s be

low

ICIL

S av

erag

e

Teac

her’s

 repo

rted

 stud

ent  

use o

f ICT

to ac

cess

exte

rnal

re

sour

ces a

nd ex

perts

that

ar

e inc

lude

d in

scal

e sco

re

T_St

UExt

#

Teac

her’s

 repo

rted

 stud

ent  u

se  of  ICT

 in  tr

adition

al  le

arning

 ta

sks t

hat a

re in

clude

d in

scal

e sco

re T_

StUT

rad#

Teac

her’s

 repo

rted

 stud

ent  u

se  of  ICT

 in  in

nova

tive  stud

ent-­‐c

entered  learning

 task

s  tha

t  are  in

clude

d  in  

scal

e sco

re T_

StUI

nno#

Wor

king

on

exte

nded

pr

ojec

ts (i

.e.

over

sev

eral

w

eeks

)

Wor

king

on

shor

t as

sign

men

ts

(i.e.

with

in

one

wee

k)

Expl

aini

ng

and

disc

ussi

ng

idea

s w

ith

othe

r st

uden

ts

Subm

ittin

g co

mpl

eted

w

ork

for

asse

ssm

ent

Wor

king

in

divi

dual

ly

on le

arni

ng

mat

eria

ls a

t th

eir o

wn

pace

Unde

rtaki

ng

open

-end

ed

inve

stig

atio

ns o

r fie

ld

wor

k

Refle

ctin

g on

thei

r le

arni

ng

expe

rienc

es

(e.g

. by

usin

g a

lear

ning

lo

g)

Plan

ning

a

sequ

ence

of

lear

ning

ac

tiviti

es fo

r th

emse

lves

Proc

essi

ng

and

anal

ysin

g da

ta

Sear

chin

g fo

r in

form

atio

n on

a to

pic

usin

g ou

tsid

e re

sour

ces

Eval

uatin

g in

form

atio

n re

sulti

ng

from

a

sear

ch

Com

mun

icat

ing

with

st

uden

ts in

ot

her

scho

ols

on

proj

ects

Seek

ing

info

rmat

ion

from

exp

erts

ou

tsid

e th

e sc

hool

Educ

atio

nal

Syst

em

Hong

Kon

g (S

AR)

ICIL

S 20

13 a

vera

geAu

stra

liaKo

rea,

Rep

. of

Tabl

e 4.1

6 Pe

rcen

tage

s of t

each

ers w

ho in

dica

te st

uden

ts o

ften

usin

g IC

T fo

r tea

chin

g ac

tiviti

es in

cla

ssro

oms

116

4.3.4  Teachers’  reported  use  of  ICT  for  various  types  of  teaching  and  learning activities (T_UseTrad, T_UseA&F, T_UseStdColl)

ICILS  also  studied  teachers’  use  of  ICT  in  various  types  of  teaching  and  learning  activities. Teachers were asked to indicate their frequency of ICT use for the

following types of activities through a 3-point Likert scale (never, sometimes, and

often):

x Presenting information through direct class instruction;

x Providing remedial or enrichment support to individual students or small

groups of students;

x Enabling student-led whole-class discussions and presentations;

x Assessing students' learning through tests;

x Providing feedback to students;

x Reinforcing learning of skills through repetition of examples;

x Supporting collaboration among students;

x Mediating communication between students and experts or external

mentors;

x Enabling students to collaborate with other students (within or outside

school); and

x Collaborating with parents or guardians in supporting students’  learning

x Supporting inquiry learning.

Table 4.17 reports the percentages of teachers who indicated often use of ICT

in the surveyed activities. From the table, only one aspect of ICT use by Hong Kong

teachers in the classroom is higher than the ICILS international average, namely

presenting information through direct class instruction (38%). All other types of

ICT usage in teaching activities by Hong Kong teachers were lower than the ICILS

international averages by 1% to 8%. The greatest differences between the Hong

Kong and ICILS average (8%) were found in two kinds of teacher use of ICT to

support   students’   collaborative   inquiry   activities:   collaboration   among   students  (8%) and supporting inquiry learning (6%), which are often considered as

important pedagogical activities to foster 21st century learning outcomes. Based on

teachers’   responses   to   this   question,  we   have   developed   three   scales   related   to  teachers’  use  of  ICT  in  teaching:  in  traditional  teaching  (T_UseTrad),  in  assessment  and giving feedback   (T_UseA&F),   and   in   supporting   students’   collaborative  inquiry (T_UseStdColl). The compositions for each of these three scales are

presented in Table 4.17.

117

38(1

.6)

9

(0.9

)

8(0

.8)

12

(1.1

)

15(1

.5)

16

(1.3

)

8(0

.9)

3

(0.5

)

5(0

.7)

3

(0.6

)

6(0

.7)

33(0

.5)

15(0

.3)

15(0

.4)

16(0

.4)

17(0

.4)

21(0

.5)

16(0

.4)

4(0

.2)

7(0

.3)

10(0

.3)

14(0

.4)

46(1

.6)S

19(0

.9)U

18(0

.9)U

10(0

.8)V

17(0

.8)

20(1

.1)

14(1

.0)V

3(0

.4)V

7(0

.6)

9(0

.7)

18(1

.0)U

42(1

.9)U

22(1

.0)U

10(1

.2)V

12(0

.7)V

15(1

.7)

20(2

.0)

8(1

.0)V

5(0

.9)

8(0

.8)

4(0

.8)V

10(1

.4)V

*P

erce

ntag

es re

flect

teac

hers

who

hav

e in

dica

ted

"Ofte

n".

#Th

ese

scal

e sc

ores

for H

K w

ere

cons

truct

ed b

ased

on

conf

irmat

ory

fact

or a

naly

ses

resu

lts o

n th

e H

K d

ata.

( )

Sta

ndar

d er

rors

app

ear i

n pa

rent

hese

s. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

SM

ore

than

10

perc

enta

ge p

oint

s ab

ove

ICIL

S a

vera

geU

Sig

nific

antly

abo

ve IC

ILS

ave

rage

VS

igni

fican

tly b

elow

ICIL

S a

vera

geT

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

belo

w IC

ILS

ave

rage

Med

iatin

g co

mm

unic

atio

n be

twee

n st

uden

ts

and

expe

rts

or e

xter

nal

men

tors

Enab

ling

stud

ents

to

colla

bora

te

with

oth

er

stud

ents

(w

ithin

or

outs

ide

scho

ol)

Col

labo

rati

ng w

ith

pare

nts

or

guar

dian

s in

su

ppor

ting

stud

ents’  

lear

ning

Supp

ortin

g in

quiry

le

arni

ng

Teac

her’s

 repo

rted

 use

 of  ICT  for  

asse

ssm

ent a

nd fe

edba

ck th

at a

re

incl

uded

in s

cale

sco

re T

_Use

A&

F#

Teac

her’s

 repo

rted

 use

 of  ICT  to  sup

port  stude

nts'  

colla

bora

tive

inqu

iry th

at a

re in

clud

ed in

sca

le s

core

T_

Use

StdC

oll#

 Tea

cher’s  re

ported

 use

 of  ICT  in  

trad

ition

al te

achi

ng th

at a

re in

clud

ed

in s

cale

sco

re T

_Use

Trad

#

Pres

entin

g in

form

atio

n th

roug

h di

rect

cla

ss

inst

ruct

ion

Prov

idin

g re

med

ial o

r en

richm

ent

supp

ort t

o in

divi

dual

st

uden

ts o

r sm

all g

roup

s of

stu

dent

s

Enab

ling

stud

ent-l

ed

who

le-c

lass

di

scus

sion

s an

d pr

esen

tatio

ns

Ass

essi

ng

stud

ents

' le

arni

ng

thro

ugh

test

s

Prov

idin

g fe

edba

ck to

st

uden

ts

Rei

nfor

cing

le

arni

ng o

f sk

ills

thro

ugh

repe

titio

n of

ex

ampl

es

Supp

ortin

g co

llabo

ratio

n am

ong

stud

ents

Educ

atio

nal

Syst

em

Hon

g K

ong

(SAR

)IC

ILS

201

3 av

erag

eAu

stra

liaK

orea

, Rep

. of

Tabl

e 4.1

7 Pe

rcen

tage

s of t

each

ers o

ften

usin

g IC

T fo

r tea

chin

g pr

actic

es in

cla

ssro

oms

118

4.3.5  Teachers’  emphasis  on  developing  students’  information  literacy (T_EMPStIL)

In   the   ICILS   teacher   questionnaire,   teachers’   emphasis   (strong,   some,   little,   no  emphasis) on developing the following ICT-based capabilities among students was

investigated:

x Accessing information efficiently;

x Sharing digital information with others;

x Using computer software to construct digital work products (e.g.

presentations, documents, images and diagrams);

x Evaluating their approach to information searches;

x Providing digital feedback on the work of others (such as classmates);

x Exploring a range of digital resources when searching for information;

x Providing references for digital information sources;

x Understanding the consequences of making information publically

available online;

x Evaluating the relevance of digital information;

x Displaying information for a given audience/purpose;

x Evaluating the credibility of digital information; and

x Validating the accuracy of digital information.

Table 4.18 reports percentages of teachers who indicated placing strong or

some emphasis on developing the above listed ICT-based capabilities among

students. The results show that the percentages of Hong Kong teachers who

reported giving   at   least   some   emphasis   to   developing   students’   ICT-based

capabilities were all lower than the ICILS international average in all of the 12

surveyed aspects, with the differences varying between 5% to 20%.

The largest differences are found in teacher’s   emphasis   on   developing  student’s   ICT-based capabilities for (1) exploring a range of digital resources when searching for information (33% for Hong Kong compared to the international mean

of 53%); (2) evaluating the relevance of digital information (36% compared to 52%); and

(3) evaluating the credibility of digital information (also 36% compared to 52%). Only

two out of the list of ICT-based capabilities were reported as emphasized by more

than 50% of Hong Kong teachers, namely, accessing information efficiently (53%) and

using computer software to construct digital work products (51%).

In contrast, more than 50% of Australian teachers reported giving emphasis

to the development of 11 out of the 12 ICT-based capabilities surveyed (the only

exception was providing digital feedback on the work of others (such as classmates), at

28%). For Korean teachers, more than 50% of them reported giving emphasis to

helping their students to develop 9 out of the list of 12 surveyed ICT-based

capabilities).

119

53(1

.7)

38

(1.8

)

51(1

.6)

36

(1.5

)

27(1

.5)

33

(1.6

)

40(1

.4)

45

(2.0

)36

(1.6

)

42(1

.5)

36

(1.6

)

36(1

.5)

63

(0.5

)43

(0.5

)56

(0.5

)48

(0.5

)34

(0.5

)53

(0.5

)49

(0.5

)51

(0.5

)52

(0.5

)54

(0.5

)52

(0.5

)51

(0.5

)76

(1.0

)S

53(1

.3)U

72(1

.1)S

53(1

.1)U

28(1

.7)V

62(1

.1)U

58(1

.3)U

51(1

.6)

66(0

.9)S

70(1

.0)S

62(1

.0)U

58(0

.9)U

62(1

.4)

50(1

.4)U

54(1

.7)

48(2

.8)

40(1

.5)U

57(1

.2)U

56(1

.1)U

47(1

.1)V

55(1

.5)

50(1

.3)V

51(1

.8)

50(1

.6)

*P

erce

ntag

es re

flect

teac

hers

who

hav

e in

dica

ted

"Ofte

n".

#Th

ese

scal

e sc

ores

for H

K w

ere

cons

truct

ed b

ased

on

conf

irmat

ory

fact

or a

naly

ses

resu

lts o

n th

e H

K d

ata.

( )

Sta

ndar

d er

rors

app

ear i

n pa

rent

hese

s. B

ecau

se s

ome

resu

lts a

re ro

unde

d to

the

near

est w

hole

num

ber,

som

e to

tals

may

app

ear i

ncon

sist

ent.

SM

ore

than

10

perc

enta

ge p

oint

s ab

ove

ICIL

S a

vera

geU

Sig

nific

antly

abo

ve IC

ILS

ave

rage

VS

igni

fican

tly b

elow

ICIL

S a

vera

geT

Mor

e th

an 1

0 pe

rcen

tage

poi

nts

belo

w IC

ILS

ave

rage

 Tea

cher’s  re

ported

 emph

asis  on  de

veloping

 stud

ents'  infor

mation  literacy  

that

are

inclu

ded

in sc

ale

scor

e T_

EMPS

tIL#

Unde

rsta

ndi

ng th

e co

nseq

uen

ces

of

mak

ing

info

rmat

ion

publ

ical

ly

avai

labl

e on

line

Valid

atin

g th

e ac

cura

cy o

f di

gita

l in

form

atio

n

Acce

ssin

g in

form

atio

n ef

ficie

ntly

Eval

uatin

g th

e re

leva

nce

of d

igita

l in

form

atio

n

Disp

layi

ng

info

rmat

ion

for a

gi

ven

audi

ence

/pu

rpos

e

Eval

uatin

g th

e cr

edib

ility

of

dig

ital

info

rmat

ion

Prov

idin

g di

gita

l fe

edba

ck o

n th

e w

ork

of

othe

rs (s

uch

as

clas

smat

es)

Expl

orin

g a

rang

e of

di

gita

l re

sour

ces

whe

n se

arch

ing

for

info

rmat

ion

Prov

idin

g re

fere

nces

fo

r dig

ital

info

rmat

ion

sour

ces

Eval

uatin

g th

eir

appr

oach

to

in

form

atio

n se

arch

es

Shar

ing

digi

tal

info

rmat

ion

with

oth

ers

Usin

g co

mpu

ter

softw

are

to

cons

truct

di

gita

l wor

k pr

oduc

ts

(e.g

. pr

esen

tatio

ns,

docu

men

ts,

imag

es a

nd

diag

ram

s)

Educ

atio

nal

Syst

em

Hon

g K

ong

(SAR

)IC

ILS

201

3 av

erag

eAu

stra

liaK

orea

, Rep

. of

Tabl

e 4.1

8 Percentages  of  teachers  put  strong  or  some  em

phasis  to  develop  students’  IC

T-b

ased

120

Based   on   the  Hong  Kong   teachers’   responses   to   eight   of   the   items   in   this  question,  we  were  able  to  construct  one  scale  on  teachers’  emphasis  on  developing  students’  information  literacy,  T_EMPStIL, as indicated in Table 4.18.

4.4 School Level Factors and Hong  Kong  Student’s  CIL  Achievement

In the previous sections, we have reported on the school level conditions,

principals’   leadership   practices   and   teachers’   ICT-using pedagogy in schools.

These factors are not independent of each other. In order to explore the plausible

influence  of  the  totality  of  school  effects  on  student’s  CIL  achievement,  multilevel  analysis using HLM has been conducted.

Usually, teacher factors are considered as nested within schools so that data

from the teacher questionnaire would be treated as level 2 factors (students (level

1) within the same class taught by the same teacher (level 2)), and data from the

principal questionnaire treated as level 3. However, as mentioned earlier, the

sampling of students in ICILS 2013 was conducted randomly among the entire

population of grade 8 students within the same school, there is no information

about the relationship between the sampled students and the sampled teachers.

Hence,  in  the  multilevel  analysis,  the  teachers’  responses  from  the  same  school are

averaged to form school level variables, even though the content is about teachers.

In summary, there are only two levels of variables in the multilevel modelling

reported in this section: student and school.

Table 4.19 lists the set of 25 variables derived from the principal and teacher

surveys   that   were   used   in   the   multilevel   analysis:   14   principal’s   e-Learning

leadership practice variables (P_EXPTPK, P_MONSULRN, P_MONSUADV,

P_RESICTM,  P_RESICTU,  P_ProvStACC,  P_TPDpart),  and  11  teachers’  ICT-using

pedagogy variables (T_EFGen, T_EFAdv, T_EFPeda, T_USEPedaT, T_StUInno,

T_StUTrad, T_StUExt, T_UseTrad, T_UseA&F, T_UseStdColl, T_EMPStIL). A list

of these variables, their meanings in brief and the relevant section where these are

presented are summarized in Tables 19a and 19b.

4.4.1 School level factors and overall CIL score

This section aims to investigate further how the various principal- and teacher-

related factors at the school level contribute to the CIL outcomes of Hong Kong

students. Specifically, the following two research questions are explored:

1. Do any of the scale variables constructed from the principal and teacher

surveys relate significantly to Hong   Kong   students’   CIL   achievement  scores?

2. What percentages of the total between-school and within-school variance

can these scale variables account for?

121

Table 4.19a List of key school-level variables* derived from the principal

Principal’  e-Learning Leadership Practice factors derived from the school questionnaire

Variable name Variable label (construct) Section in chapter 4

P_EXPTPK Extent to which the principal expects teachers to possess pedagogical knowledge and skills in e-Learning

4.2.2

P_MONSULRN Extent to which the school  monitors  teachers’  ICT use for students' learning

4.2.3

P_MONSUADV Extent to which the school  monitors  teachers’  ICT use for developing students' advanced ICT skills

P_RESICTM Extent to which the principal takes responsibility for ICT soft and hardware management 4.2.4

P_RESICTU Extent to which the principal takes responsibility for e-Learning curriculum issues

P_ProvStACC Extent to which the school makes provisions for student access to computers in and out of school 4.2.5

P_TPDpart Extent  of  teachers’  participation  in  professional  development as reported by the principal. 4.2.6

P_PriAcComp Priority given to improve access to computers and internet in school

4.3.1 P_PriELRes Priorities given to increase the range of e-Learning resources in school

P_PriPedaUse Priorities given to improve supports for teachers on pedagogical use of ICT in school

NP_ObsInsufH&S

The extent to which insufficient ICT hard and software  hinder  the  school’s  capacity  to  realize  its e-Learning goals

4.3.2

NP_ObsbyTs The extent to which obstacles faced by teachers hinder  the  school’s  capa-city to realize its e-Learning goals

NP_ObsCurr The extent to which curriculum and assessment related  obstacles  hinder  the  school’s  capacity  to  realize its e-Learning goals

NP_InsufRes The extent to which insufficient budget, space, or  mismatch  of  goals  hinder  the  school’s  capacity to realize its e-Learning goals

122

Table 4. 19b List of key school-level variables* derived from the teacher questionnaire

Teacher’s  ICT-using Pedagogy factors derived from the teacher questionnaire

Variable name Variable label (construct) Section in chapter 4

T_EFGen Teacher’s  self-efficacy in general ICT tasks

4.4.1 T_EFAdv Teacher’s  self-efficacy in advanced ICT skills

T_EFPeda Teacher’s  self-efficacy in pedagogical use of ICT

T_USEPedaT Teacher’s  reported  use  of  pedagogical  ICT  tools 4.4.2

T_StUInno Teacher’s  reported  student  use  of  ICT  in  innovative student-centered learning tasks

4.4.3 T_StUTrad Teacher’s  reported student use of ICT in

traditional learning tasks

T_StUExt Teacher’s  reported  student  use  of  ICT  to  access  external resources and experts

T_UseTrad Teacher’s  reported  use  of  ICT  in  traditional  teaching

4.4.4 T_UseA&F Teacher’s  reported  use  of ICT for assessment

and feedback

T_UseStdColl Teacher’s  reported  use  of  ICT  to  support  students' collaborative inquiry

T_EMPStIL Teacher’s  reported  emphasis  on  developing  students' information literacy 4.4.5

Similar to the procedure described in section 3.5, the first step in multilevel

modelling is to conduct a variance component analysis to obtain information on

the Null model about the total variances at the student and school levels.

Conceptually, if all students were randomly assigned to all schools such that all

schools will have the same mean CIL, then all the variations in CIL score would be

within-school variations. At the other extreme, if all students were strictly ranked

according to their CIL scores and then assigned sequentially to schools strictly

according to this rank ordering, then each school would be as homogenous as

possible  in  students’  CIL  achievement  and  all  variations  in  students’  CIL  score  will  be predicted by the between-school variations. In real life, the distribution of the

variance across within-school and between-school variances depends very much

on the homogeneity of schools within the specific system, and hence varies greatly

across countries.

Results returned by HLM under the null model show that the within-school

(level 1) variance is 4814.74, while the between school (level 2) variance is 5072.11.

This  means  that  more  than  half  of  the  total  variance  in  students’  CIL  achievement  (51.3%) is between-school variations. In fact, Hong Kong has amongst the highest

percentage of between-school variance (Fraillon et al, 2014). This means that the

123

school a student  attends  is  a  strong  predictor  of  the  students’  CIL  outcome,  and  that Hong Kong schools differ greatly from each other in this respect. It is well

known through other international comparative studies of student achievement

such as TIMSS, PIRLS and PISA that Hong Kong schools are highly non-

homogenous, due to highly differentiated student in-take. At the same time, school

level factors such as leadership and pedagogy would also contribute to variations

in  students’  achievement.  This  is  explored  using  Random Intercepts Fixed Slope

modelling using all the variables listed in Table 4.19 as level 2 predictor variables.

The HLM results for the analysis after reaching convergence is presented in Table

4.20.

The results in Table 4.20 show that only four variables remain as statistically

significant predictors  of  students’  CIL  scores  at the end of the modelling process,

two of which are scales derived from the principal questionnaire on obstacles

encountered in the implementation of e-learning and two scales related to ICT use

from the teacher questionnaire. Of these four scale factors, two have positive

coefficients:   teachers’   reported   student   use   of   ICT   in   traditional   learning   tasks  (T_StUTrad) with a coefficient of 21.35, and the extent to which curriculum and

assessment  related  obstacles  hinder  the  school’s  capacity  to  realize  its  e-learning

goals as reported by principals (NP_ObsCurr) with a coefficient of 18.83.

It  is  noteworthy  that  taking  all  the  various  factors  into  account,  it  is  students’  actual use of   ICT   in   learning   that   contributes  most   important   to   students’   CIL  outcomes. One may find it surprising that the other two reported types of uses of

ICT by students did not have significant coefficients, especially since the other two

types of uses (for student-centred learning tasks and for reaching out to external

resources and experts) are likely to contribute even more to CIL outcomes. The

most likely reason is that these two types of ICT use by students are relatively

infrequent and so the variation across a low average is not large enough to produce

a statistically significant coefficient. In fact, Table 4.20 shows that student ICT use

in student-centered learning tasks also has a sizeable positive correlation of 16.94,

although it is not statistically significant (p=0.16).

It may look counter-intuitive that a reported obstacle, NP_ObsCurr, has a

positive coefficient, which means that the more strongly the principal reported

curriculum and assessment related obstacles, the higher the CIL score of the

students. We interpret this result to indicate that the principals who reported this

type of obstacles more strongly are those who have really given more thoughts to

the necessary changes in curriculum and assessment practices in order to realize

the school’s  e-learning goals. Such awareness at the leadership level should also

bring  different   implementation  practices   that   influence   students’  CIL  outcomes,  though these other changes at the school may not be captured by the principal

questionnaire.

124

Table 4.20 Multilevel model res0.0ults  for  students’  CIL  scores  using  school-level variables as level 2 predictors (z-scores)

There are two factors that have significant negative coefficients. One is the

extent to which insufficient ICT hardware   and   software   hinders   the   school’s  capacity to realize its e-Learning goals, NP_ObsInsufH&S, with a coefficient of -15.88.

Variables Brief description Coefficient (Error) P-ValueINTERCEPT 507.23 (6.33) 0.00

P_MONSULRN P: monitors ICT use for students' learning - - -

P_MONSUADV P: monitors ICT use for students' advanced ICT skills

8.31 (6.47) 0.20

P_RESICTM P: takes responsibility for ICT management - - -

P_RESICTU P: takes responsibility for e-Learning curriculum issues

-12.78 (6.62) 0.06

P_MONStACC P: makes provisions for student ICT access -9.97 (6.04) 0.10

P_TPDpart P: T participation in ICT-related professional development

10.40 (7.43) 0.17

P_PriAcComp P priority: improve ICT access in school - - -

P_PriELRes P priority: increase e-Learning resources - - -

P_PriPedaUse P priority: improve pedagogical support for T - - -NP_ObsInsufH&S P obstacle: insufficient ICT infrastructure -15.88 (6.46) 0.02

NP_ObsbyTs P obstacle: Teacher factors - - -

NP_ObsCurr P obstacle: curriculum & assessment related 18.83 (6.84) 0.01

NP_InsufRes P obstacle: Insufficient resources -12.52 (7.04) 0.08

T_EFGen T: general ICT self-efficacy - - -

T_EFAdv T: advanced ICT self-efficacy - - -

T_EFPeda T: pedagogical ICT self-efficacy 11.95 (6.47) 0.07

T_USEPedaT T: pedagogical use of ICT tools -21.91 (8.71) 0.01

T_StUInno T: student ICT use in student-centered learning tasks

16.94 (11.93) 0.16

T_StUTrad T: student ICT use in traditional learning tasks 21.35 (8.39) 0.01T_StUExt T: student ICT use to reach out - - -

T_UseTrad T: ICT use in traditional teaching - - -

T_UseA&F T: ICT use for assessment and feedback - - -

T_UseStdColl T: ICT use to support students' collaborative inquiry

-14.94 (9.15) 0.11

T_EMPStIL T: emphasis on developing students' IL -10.35 (10.28) 0.32

Between School Variance & Percentage 5072.11 51.3%

Within School Variance & Percentage 4814.74 48.7%

Between School 1924.06 37.93%

Within School 0.01 0.00%

* All independent variables were transformed into zscores# The dependent variables (i.e. 5 plausible values) were in their original score format (Mean=509, SD=7.4 for HK sample) ̂The dependent variable (i.e. CIL aspect scores) were presented in chapter 2.6.

"Variance explained" refers to the percentage of variance the set of variables in the model has accounted for level 2 total variance."-" indicates variables that were not input into the specific model due to negligibly small effects on students' CIL scores (the outcome variable).Bolded variables are significant with p-value <=0.05

Variance explained by

model and their

Model: School level factors and overall CIL score#

Null Model

125

This finding is reasonable. It is important to note that this scale factor is

different in nature from the actual level of ICT resources available in schools. A

principal in a relatively less well–resourced school in terms of ICT infrastructure

may not report this as an obstacle if the demand/desire to use the facilities for

teaching and learning is low. Conversely, the principal of a relatively well-

resourced school may still find hardware and software availability to be a serious

obstacle if most of the teachers are actively seeking to leverage the use of ICT for

student learning.

Another factor with a negative coefficient (-21.91) is the teachers’   reported  use of pedagogical ICT tools, T_USEPedaT. This is an interesting contrast to the

factors  on  students’  ICT  use  for  learning.  Our  interpretation  of  this  finding  is  that  the negative coefficient may not be related to ICT use per se, but that this scale

score is highly correlated with a teacher-centred, traditional pedagogy, which is

not  conducive  to  students’  development  of  CIL.  

Examining the variance explained by this model, we find that this model

explains 37.93% of the total between-school variance in CIL score, which is a

substantial proportion of the between-school variance. On the other hand, the

within-school variance explained is 0.00%, which is appropriate as all the factors

considered operate at the school level only.

4.4.2 School level factors and the seven CIL aspect scores

Similar to our exploration in section 3.5.2, we are interested in investigating the

plausible   influence   of   school   factors   on  Hong  Kong   students’   seven  CIL   aspect  scores. Again, Random Intercepts Fixed Slope modelling (using HLM) was used

for each CIL aspect to discover:

1. Do any of the context variables collected through the student survey

significantly influence each of their seven CIL aspect scores?

2. What percentage of the total between-school and within-school variance

can these variables account for?

In order to be comparable to the model presented in the previous section, the

same set of principal e-Learning leadership factors (14 variables) and teacher ICT-

usage pedagogy factors (11 variables) were used in each of the models created to

analyse the plausible influence of school level factors on the seven aspects of CIL.

Once again, these models face exactly the same set of limitations that originated

from the design of the test booklets described in section 3.5.2. Hence, results

reported here should be interpreted with caution and are for exploratory purposes

only.

Among the 25 school level factors investigated, the four factors having

significant coefficients in the model for overall CIL achievement scores as reported

in the previous section (NP_ObsInsufH&S, T_USEPedaT, NP_ObsCurr, and

126

T_StUTrad) were found to be significant in more CIL aspect models than other

school level factors. First of all, the factor NP_ObsInsufH&S, indicating the lack of

hardware and software as an obstacle, is significant in all seven models with

negative coefficients ranging from -0.02 to -0.05. The second most important factor

is  teachers’  reported  use  of  pedagogical  ICT  tools  (T_USEPedaT).  All  coefficients  for this factor, similar to the model in 4.4.1, are negative and are statistically

significant in five CIL aspect models, with the exception of the models of creating information (p=.09) and using information securely and safely (p=.07).

Curriculum and assessment related obstacles (NP_ObsCurr), as reported by

principals, is another important factor having positive coefficients for four CIL

aspects, namely assessing and evaluating information, transforming information, creating information, and using information securely and safely.

In addition, the teacher factor that reported student use of ICT in traditional

learning tasks (T_StUTrad) has positive significant coefficients in models of three

CIL aspects, including assessing and evaluating information, creating information, and

sharing information.

Last but not least, similar to the multi-level analysis results for the overall CIL

score, school level factors can only explain a substantial percentage of the between-

school variance but not the within-school variance in student achievement for all

seven CIL aspects.

4.5 Summary

In this chapter, school level factors collected through principal and teacher

questionnaires have been reported. Overall, Hong Kong schools had better than

the ICILS international average level of ICT infrastructure and resources, but the

level of ICT use for teaching and learning were still low. In particular, the

opportunities for students to use ICT for learning were much lower than their

international counterparts, especially in ICT use for student-centered learning

tasks. Teachers generally had high confidence in their general use of ICT, but lower

in their own pedagogical ICT competence.

We have also investigated whether and how these factors associate with

student CIL scores. ICT infrastructure and resources in Hong Kong schools,

principal’s   e-Learning   leadership   practices,   as   well   as   teachers’   ICT-using

pedagogy have been reported and explored in different sections of this chapter.

Models generated from multi-level analysis suggest that the effect of school level

factors   on   Hong   Kong   students’   CIL   scores   as   the   independent   variables  corroborate   the   importance   of   students’   use  of   ICT   in   traditional   learning   tasks  (T_StUTrad, coefficient=21.35), and principals’   awareness  of the curriculum and

assessment related issues (obstacles) involved in the realization of e-learning goals

127

CoefficientError

P-Value

CoefficientError

P-Value

CoefficientError

P-Value

CoefficientError

P-Value

CoefficientError

P-Value

CoefficientError

P-Value

CoefficientError

P-Value

0.70(0.01)

0.00

0.37(0.01)

0.00

0.42(0.02)

0.00

0.3

6(0

.01

)0

.00

0.5

3(0

.02

)0

.00

0.3

9(0

.01

)0

.00

0.5

7(0

.01

)0

.00

--

---

---

--

--

--

--

--

--

---

---

---

--

--

--

--

--

0.0

2(0

.01

)0

.02

--

---

---

--

--

--

--

--

--

--0.02(0.01)

0.05

-0.02(0.01)

0.10

-0.03(0.01)

0.03

-0.0

2(0

.01

)0

.13

-0.0

2(0

.02

)0

.18

-0.0

1(0

.01

)0

.25

-0.0

2(0

.01

)0

.10

-0.02(0.01)

0.01

-0.02(0.02)

0.10

-0.020.01

0.19

-0.0

3(0

.01

)0

.04

-0.0

3(0

.02

)0

.06

-0.0

3(0

.01

)0

.02

--

-0.01(0.01)

0.16

--

-0.020.02

0.33

0.0

2(0

.01

)0

.07

0.0

4(0

.02

)0

.05

0.0

2(0

.01

)0

.12

--

---

---

--0.020.02

0.16

--

--

--

-0.0

2(0

.01

)0

.08

--

---

---

---

--

--

--

--

--

--

-0.03(0.01)

0.01

0.02(0.02)

0.31

0.030.02

0.06

--

--

--

0.0

2(0

.01

)0

.08

0.0

1(0

.01

)0

.19

-0.02(0.01)

0.05

-0.05(0.01)

0.00

-0.04(0.01)

0.01

-0.0

4(0

.01

)0

.01

-0.0

4(0

.02

)0

.01

-0.0

4(0

.01

)0

.00

-0.0

3(0

.01

)0

.00

--

---

---

--

--

--

--

--

--

-0.02(0.01)

0.08

0.05(0.02)

0.00

0.030.02

0.09

0.0

4(0

.01

)0

.01

0.0

4(0

.02

)0

.02

0.0

2(0

.01

)0

.08

0.0

4(0

.01

)0

.00

-0.02(0.01)

0.02

-0.03(0.02)

0.10

-0.020.02

0.23

-0.0

2(0

.01

)0

.08

-0.0

4(0

.02

)0

.06

--

--0

.02

(0.0

1)

0.1

1--

-0.03(0.02)

0.07

--

--

--

0.0

2(0

.02

)0

.28

--

--

--

--

---

---

--

--

--

--

--

--

---

---

---

--

--

--

--

--

--

-0.01(0.01)

0.20

0.02(0.02)

0.30

0.030.01

0.05

0.0

2(0

.01

)0

.07

0.0

3(0

.02

)0

.10

0.0

3(0

.01

)0

.03

0.0

1(0

.01

)0

.22

-0.02(0.01)

0.02

-0.04(0.02)

0.04

-0.05(0.02)

0.01

-0.0

5(0

.01

)0

.00

-0.0

4(0

.02

)0

.09

-0.0

4(0

.01

)0

.00

-0.0

2(0

.01

)0

.07

--

-0.07(0.04)

0.07

0.05(0.02)

0.02

0.0

3(0

.02

)0

.15

0.0

5(0

.03

)0

.15

--

-0

.03

(0.0

2)

0.1

50.02(0.01)

0.10

0.05(0.02)

0.03

--

-0

.03

(0.0

2)

0.0

50

.08

(0.0

2)

0.0

00

.03

(0.0

2)

0.0

30

.02

(0.0

1)

0.1

4--

--0.02(0.02)

0.16

--

--

--

-0.0

3(0

.02

)0

.13

--

--

--

--

---

---

--

--

--

--

--

-0.0

2(0

.01

)0

.11

0.03(0.02)

0.09

--

-0.030.02

0.15

0.0

2(0

.02

)0

.18

--

-0

.03

(0.0

2)

0.1

5-

--

-0.02(0.01)

0.07

-0.05(0.02)

0.07

-0.05(0.02)

0.03

-0.0

4(0

.02

)0

.02

-0.0

5(0

.02

)0

.05

-0.0

3(0

.02

)0

.13

--

--0.02(0.01)

0.15

-0.03(0.02)

0.15

--

--

--

-0.0

4(0

.03

)0

.13

--

--0

.02

(0.0

1)

0.1

2

Betw

een

Sch

oo

l V

ari

an

ce

& P

erc

en

tag

e0.0

067

11.7

%0.0

277

29.6

%0.0

147

7.9

%0.0

182

29.1

%0.0

345

39.9

%0.0

104

7.2

%0.0

094

15.5

%

Wit

hin

Sch

oo

l V

ari

an

ce

& P

erc

en

tag

e0.0

505

88.3

%0.0

659

70.4

%0.1

711

92.1

%0.0

444

70.9

%0.0

519

60.1

%0.1

340

92.8

%0.0

515

84.5

%

Betw

een

Sch

oo

l0.0

030

45.0

1%

0.0

116

41.8

3%

0.0

061

41.5

5%

0.0

075

41.3

1%

0.0

131

38.0

3%

0.0

057

54.1

2%

0.0

036

37.8

2%

Wit

hin

Sch

oo

l0.0

000

0.0

0%

0.0

000

0.0

0%

0.0

000

0.0

0%

0.0

000

0.0

0%

0.0

000

0.0

0%

0.0

000

0.0

0%

0.0

000

0.0

0%

* A

ll in

dep

en

den

t va

riab

les w

ere

tra

nsfo

rmed

in

to z

score

s# T

he d

ep

en

den

t va

riab

les (

i.e.

5 p

lau

sib

le v

alu

es)

were

in

th

eir o

rig

inal score

form

at

(Mean

=509,

SD

=7.4

for

HK

sam

ple

)^ T

he d

ep

en

den

t va

riab

le (

i.e.

CIL

asp

ect

score

s)

were

pre

sen

ted

in

ch

ap

ter

2.6

. "V

arian

ce e

xpla

ined

" re

fers

to t

he p

erc

en

tag

e o

f va

rian

ce t

he s

et

of

variab

les in

th

e m

od

el h

as a

ccou

nte

d f

or

leve

l 2 t

ota

l va

rian

ce.

"-"

ind

icate

s v

ariab

les t

hat

were

not

in t

he m

od

el d

ue t

o n

eg

ligib

ly s

mall e

ffects

on

stu

den

ts' C

IL s

core

s (

the o

utc

om

e v

ariab

le).

Variab

les h

igh

ligh

ted

hh

ave

p-v

alu

e <

=0.0

5

Mo

del:

Sch

oo

l le

vel fa

cto

rs &

th

e 7

CIL

asp

ect

sco

res^

Mo

del:

Sch

oo

l le

vel fa

cto

rs a

nd

th

e 7

CIL

asp

ect

sco

res^

Asp

ect

2.1

Tra

nsfo

rmin

g

Info

rmati

on

Asp

ect

2.2

C

reati

ng

in

form

ati

on

Asp

ect

2.3

S

hari

ng

In

form

ati

on

Asp

ect

2.4

U

sin

g in

form

ati

on

secu

rely

& s

afe

ly

Asp

ect

1.1

K

no

win

g a

bo

ut

an

d

un

ders

tan

din

g c

om

pu

ter

use

Asp

ect

1.2

A

ssessin

g &

Ev

alu

ati

ng

In

form

ati

on

Asp

ect

1.3

M

an

ag

ing

In

form

ati

on

Vari

an

ce

exp

lain

ed

b

y

mo

del

an

d t

heir

p

erc

en

tag

es

INTRCPT

P_MONSU

LRN

P_MONSU

ADV

P_RESICTM

P_RESICTU

P_MONStACC

P_TPDpart

P_PriAcComp

P_PriELRes

P_PriPedaUse

NP_ObsInsufH&S

NP_ObsbyTs

NP_ObsCurr

NP_InsufRes

P_EXPTPK

T_EMPStIL

Nu

ll M

od

el

T_StUTrad

T_StUExt

T_UseTrad

T_UseA&F

T_UseStdColl

T_EFG

en

T_EFA

dv

T_EFPeda

T_USEPedaT

T_StUInno

Tabl

e 4.2

1 M

ultil

evel

mod

el re

sults

for e

ach

of th

e sev

en st

anda

rdiz

ed C

IL a

spec

t sco

res u

sing

scho

ol le

vel

vari

able

s as l

evel

2 p

redi

ctor

s

128

(NP_ObsCurr, coefficient=18.83) are the two statistically significant positive contributors  to  students’  CIL  outcomes. The fact that both of the significant positive school level factors are not directly related to ICT resources nor ICT-using pedagogy in schools flags that Hong Kong schools should also take initiatives to integrate ICT into parts of the school curriculum previously not explored with ICT-using pedagogy.

The analysis also found two factors with significant negative coefficients. Teachers’  reported  use  of  pedagogical  ICT  tools  (T_USEPedaT,  coefficient=-21.91), which can also be referred to as e-teaching, had a negative contribution to students’  CIL. Another negative factor was the extent to which insufficient ICT hardware and   software   hinders   the   school’s   capacity   to   realize   its   e-Learning goals (NP_ObsInsufH&S, coefficient=-15.88).

Further multilevel analysis of the influence of principal and teacher factors on each of the seven CIL aspect scores show findings that are similar to those suggested by the multilevel model on overall CIL scores. These findings have important implications for policy and practice related to e-learning if the development of CIL is an important goal.

129

Chapter 5

Learning & Assessment Designs to foster students’ CIL

The findings from ICILS 2013 reported in the earlier chapters show that the most

important  influences  on  Hong  Kong  students’  CIL  outcomes  are  those  at  the  school  level. Among these influences, the single most important factor is the opportunities

to use ICT in learning that teachers provide to students. Furthermore, the findings

show that having access to computers and the Internet, as well as using them for

personal and social communication purposes  per   se  do  not  affect   students’  CIL  outcomes.

Another   important   finding   from   ICILS   2013   is   that   neither   the   teachers’  reported use of ICT for teaching, nor the extent of their self-reported emphasis on

developing   students’   information   literacy   has any significant relationship with

their  students’  CIL  scores.   In  the  evaluation  study  of   the  e-Learning Pilot Study

that CITE conducted for the Education Bureau (CITE, 2014), five out of the 61 pilot

schools were also among the sampled ICILS schools. Analysis shows that the

distribution of scores for these five schools is not outstanding compared to the

overall performance of the Hong Kong schools. Qualitative findings from that

study also show that even teachers in the IT pilot schools generally do not

understand the exact meaning of CIL, and have very little knowledge of how to

design   learning   tasks   that  would   foster   students’  CIL  and  of  how   to  assess  CIL  outcomes.

In this chapter, we draw on a number of design-based research and

development projects that CITE has undertaken with Hong Kong schools to foster

students’  information  literacy,  inquiry  and  self-directed learning skills in order to

identify design principles and sample designs for learning and assessment of CIL.

This chapter firstly introduces several design principles. It then shows sample

learning designs of varying complexity in different subject areas.

5.1 Principles of Learning Design for CIL Development

With the support of appropriate digital technology, learning can take place

anywhere and anytime. This has made learning a continuous process and lifelong

learning skills a key competence for the 21st century. CIL is the basic competency

underpinning  ‘learning  how  to   learn’  (ALA,  1989;  Bundy,  2004).  Studies  of  ICT-

enabled innovative pedagogical practices show that the extent to which students

130

are able to demonstrate 21st century skills in the process of learning depends on whether certain critical characteristics are found in the learning activities they engage in (Law, Lee and Chow, 2002; SRI, 2011). Students are much more likely to develop CIL if they engage in solving real-world problems in collaboration with their peers. The following are the key design principles that these studies identified for  fostering  students’  CIL:  

x Priority for  ICT  use  is  given  to  students’  learning,  not  teaching; x Learning activities are learner-centric and inquiry- oriented; x Learning tasks are extended in time, comprising multiple stages and/or

parts, with interim products generated in the process; x Learning activities are authentic and related to students' daily life

experiences; x Learning tasks are open-ended, providing opportunities for students to

make judgements; and x Provide learners with the opportunity to use ICT to access different

sources of information, organize, compare and contrast, analyse and integrate information.

5.2 Principles of Assessment Design

Traditionally, assessment is perceived and designed as a step that takes place at the end of the learning process for summative purposes. This concept of assessment is inadequate to evaluate learning outcomes, which are not static knowledge and skills, but capacities, such as CIL, that enable students to dynamically handle new situations and problems.

Contemporary curriculum theories advocate the concept  of  “assessment  as  learning”  (Dann,  2002),  which considers assessment as an integral part of learning and teaching, with students actively involved in this process. Assessment as learning occurs when students reflect on and monitor their own progress to inform their formulation of future learning goals and take responsibility for their own past and future learning.

Traditional paper and pencil tests are not suitable for assessment as learning. Performance assessment that requires students to demonstrate their knowledge and skills through task completion as in ICILS is considered to be appropriate for evaluating  students’  CIL  (Knight,  2006; Helvoort, 2010). Assessment rubrics and self-evaluation checklists are two of the most commonly used instruments in performance assessment. Rubrics are essential instruments for implementing Assessment as Learning. These are descriptive scoring tools for rating authentic student work qualitatively.

131

Rubrics use specific criteria as a basis for evaluating or assessing student

performance, and provide narrative descriptions that are separated into levels of

possible performance related to a given task (Aster & McTighe, 2001; Moskal, 2000).

Many benefits associated with the use of rubric-based assessment have been

reported (Goodrich, 1997; Heman, Aschbacher & Winters, 1992; Moskal, 2000):

x It serves as a powerful communication tool among teachers, students and

parents, helping them to reach a common understanding on the criteria

and expected level of performance;

x It helps students to develop a clear understanding of what constitutes

excellence and how to evaluate their own work;

x The explicit assessment criteria helps students to develop plans for

improving their own performance; and

x It helps teachers or other raters to be accurate, unbiased and consistent in

scoring.

Table 5.1 below shows an example assessment rubric for evaluating a poster

creation task, which clearly describes the criteria for different levels of performance.

Table 5.1 Assessment  rubrics  for  evaluating  “creating  information” Level Task Novice Basic Proficient Advanced

Creating a poster

Graphics/ image is not used or is not related to the

topic

Some graphics/ images are used, but the resolution and/or size is/are not appropriate

Some relevant graphics/ images are used, most

with appropriate size and

resolution

All the graphics/ images are

relevant to the topic and with

appropriate size and resolution

Apart from rubrics, self-evaluation checklists are often used in the context of

self- and peer- evaluation. A checklist provides a list of measurable categories and

indicators for project, product and performance, allowing students to judge their

own or peer’s  performance  and  determine  whether  they  have  met  the  established  criteria of a task.

The use of checklists has been found to foster metacognitive skills, enhance

the development of appropriate learning strategies and the ability to learn how to

learn (Bransford, Brown and Cocking, 1999; Stenmark, 1993; Stergar, 2005).

Checklists are more concrete, easier to understand and operationalize, and are

especially useful for younger learners. Table 5.2 is an example of a checklist for

evaluating a poster from the creating information perspective, based on the rubric

in Table 5.1.

132

Table 5.2 An  assessment  checklist  for  “creating  information”  in  a  poster  creation  task Self-evaluation checklist-creating poster Please ✓ where appropriate

☐ The poster contains a title that clearly reflects the topic or theme

☐ The poster contain relevant and accurate information

☐ Graphic elements, such as pictures, photographs, charts, tables, scientific drawing, etc. add to the overall effectiveness of the poster

☐ There is a coherent , flowing organization to the poster with the various elements (graphic, text..) working well together

☐ The poster is skillfully designed and crafted using appropriate graphic design tools

☐ The content is not directly copied from books/ internet

☐ The fonts are easy to read and the font size varies appropriately for headings and text

☐ Graphics are of proper size and resolution and are strategically placed to enhance comprehension

5.3 Learning Designs Targeting Specific CIL Aspects

The assessment framework used in ICILS comprises two strands: collecting and managing information, and producing and exchanging information, which can be further broken down into a total of seven CIL aspects, as shown in Table 1.2. Results  from  ICILS  2013  show  that  Hong  Kong  students’  performance  is  especially  weak in three aspects: accessing and evaluating information, managing information, and sharing information. Relatively short and focused learning and assessment activities, usually in the form of an exercise, can be designed to help novice students develop competence in a specific CIL aspect.

More complex tasks set within authentic contexts, such as an extended project requiring competence in several CIL aspects operating in concert, are more effective in fostering higher levels of competence. In this section, we will present sample designs for learning and assessment that target the three weakest aspects in  Hong  Kong  students’  ICILS  performance.

5.3.1 Learning designs to foster skills for accessing and evaluating information

Accessing information With so much information that can be easily accessible through the Internet with little or no effort, it is a common observation that students are prone to direct copying of information without real understanding.

133

The following is a learning unit developed by a P.5 mathematics teacher, Ms X, with   the   purpose   of   fostering   students’   skills   in   accessing   information.   The  curriculum topic was ancient calculation tools. The teacher first asked students to work in groups of four to find information on four ancient calculation tools on the web. It was observed that students could readily locate the correct information, but tended to copy the information directly for submission. Figure 5.1 shows a sample of   students’   submission   for   this   task,  which   is  a  direct   copy-and-paste from the web.

* Source: http://edbsdited.fwg.hk/e-Learning/eng/index.php?id=4 Figure 5.1 Sample*  of  students’  submitted  work  on  ancient  calculating  tools

The teacher then introduced a follow-up task to the students: create comic

strips to introduce two of the ancient calculating tools the group identified to others using the software Toondoo.

The task nature was interesting to the children, and at the same time required them to use their own words to summarize the information to fit the word limit appropriate for the comic strip genre. Figure 5.2 shows one of the comic strips students created for this task. Here, the students were using their own words to express their ideas. Ms X was skilful in creating two interconnected tasks.

Competence in accessing information is not simply being able to find relevant information, but also being able to understand and abstract the most important information.

134

By assigning an information creation task to students, the teacher tacitly

brings home the above message to students, and provides a meaningful context for

them to conduct the summarizing information task.

It is also evident through this example that (1) even in this relatively short

learning unit, students are given the opportunity to learn two CIL aspects:

accessing and creating information; and (2) without the second, creating

information task, it would be extremely difficult to guide students to further

develop their accessing information skills in relation to their ability to summarize

information succinctly.

* Source: http://edbsdited.fwg.hk/e-Learning/eng/index.php?id=4

Figure 5.2 Sample of the comic strips created by the students

Evaluating information

To be able to evaluate information competently, students need to be able to judge

the quality, relevance, authoritativeness, bias, currency with respect to time,

coverage and accuracy of the information received.

The following are two sample learning units, at the primary and secondary

levels respectively, to foster this competence. At the primary school level, helping

students to discriminate facts from opinions is important.

Figure 5.3 shows samples of work from a P.6 student in response to a news-

reading task, which required the student to (1) find news reports on air pollution

from two newspapers, (2) list out the facts and opinions from each report, and (3)

compare which report is more objective and hence more reliable as a source of

information. Table 5.3 presents the assessment rubric for this task.

135

Table 5.3 An  assessment  rubric  for  students’  work  on  evaluating  information Level

CIL aspect Novice Basic Advanced

Evaluating information

Students are not able to distinguish facts from opinions

Students are partially able to distinguish facts from opinions

Students are able to distinguish fact and opinion

*Source: http://iltools.cite.hku.hk/exemplars.php?subject=sph_p6_gs Figure 5.3 A  P.6  student’s  work  on  evaluating  information  from  news  reports Figure 5.4 shows the work of a Secondary 3 student on a website evaluation

task connected with the curriculum topic on a balanced diet. Students were asked to find three websites on slimming (weight reduction) and to evaluate (1) whether the slimming method was effective, (2) whether the slimming method posed risks, and (3) whether the information provided was trustworthy.

136

Table 5.4 shows the checklist and assessment rubric the teacher gave to the students for their own self-evaluation after they completed the task. It is evident that the checklist and rubric was also a learning resource in giving guidance to students on how to evaluate information on the website.

*Source: http://iltools.cite.hku.hk/exemplars.php?subject=yls_bio Figure 5.4 A  Secondary  3  student’s  work  on  evaluating  information  from  websites

Some teachers introduce the concepts of direct and indirect sources of

information and arrange for students to engage in direct information gathering through field observations and interviews. The appropriate use of keywords and Boolean operators can help students to conduct more effective searches online.

137

Other useful learning activities include getting students to ask themselves the six  “W”  questions  (what,  when,  where,  who,  why  and  how),  engage  in  outlining  and brainstorming. Mind-mapping can be used in conjunction with the above activities or as a standalone activity to foster  students’  ability  to  access  and  evaluate  information. Online mind-mapping tools allow students to collaborate and share their mind maps.

Evaluation of information is relatively unfamiliar and challenging for Hong Kong students. For novice students such as primary school children, the provision of an itemized list of what needs to be done, similar to that used in Table 5.4, would be helpful. For example, in an activity to find information about a country park, primary two students were able to access the targeted information without much difficulty by using a list of items (name of the country park, the route and common plants and animals found in that park) provided by their teacher.

Table 5.4 A self-evaluation checklist and associated assessment rubric for the website evaluation task

Self-evaluation checklist-creating poster Please ✓ where appropriate

☐ I have checked the accuracy of the content, e.g. there is no wrong spelling.

☐ I have considered the relevance of this website, e.g. whether the information meets my expectation.

☐ I have considered when the information on this website was created/last updated, and whether it is up-to-date.

☐ I have considered the objectiveness of this website, e.g. whether the information provided is biased towards particular positions.

☐ I have considered the authoritativeness of this website, e.g. whether the author is an authority on this subject matter.

☐ I have considered whether complete information has been provided on the website, e.g. on authorship and publishing organization.

☐ I have considered the comprehensiveness of the information provided, e.g. supporting evidence and links to other related information.

Assessment rubric for website evaluation

Novice ☐ Basic ☐ Proficient ☐ Advanced ☐ Fails to evaluate the information on the website

Can use one criterion to evaluate the website information

Can use two criteria to evaluate the website information

Can apply more than two criteria for website evaluation

138

5.3.2 Learning designs to foster skills for managing information

Managing   information  refers   to  students’  ability   to  adopt  and  adapt  schemes  of  information classification and organization to arrange and store information for efficient use and/or re-use. Figure   5.5   shows   a   sample   of   P.2   students’  work   in   an  English   class.   The   focal  learning task was to interview some classmates about their favorite festivals.

After the interview, they had to record, analyse and organize the interview data online, give an oral presentation on their classmates' favorite festivals, and finally write an online report. In this example, the teacher skilfully designed and used Google-forms to help her students to analyse, organize and make use of their collected data.

Figure 5.5 Sample  of  P.  2  students’  work  on  managing  information Figure   5.6   presents   samples   of   students’   work   requiring   the   skills   of  

managing information collected from an experiment. In this task, P.5 students designed and conducted an experiment to compare the water absorption capacity of several brands of tissue paper. They then had to design an appropriate template and use that to record their data. As can be seen from Figure 5.6, all three groups were able to design a table for capturing the experimental data. However, there are also some observable differences.

139

Figure 5.6 Samples  of  P.  5  students’  template  for  recording  experimental  data

140

Group 1 had a novice approach to measurement and considered the time necessary for absorbing a given amount of water as the only criterion for ranking the performance of the three brands of tissue paper. Group 2 identified the need to compare the designed absorption capacity as indicated on the package against the measured absorption capacity for each brand. Group  3’s  design  was  a  variation  of  group   1’s.   They   considered  when   a   piece   of   tissue   paper   could   not   absorb   the  designated amount of water, and used the number of tissue papers needed to completely  absorb  the  “spilt  water”  as  the  needed  measurement.  

It is clear from this example that when the teacher opened up the decision on what to measure to the students, new opportunities became available to the students to learn from reviewing the answers of others.

In this example, the teacher further extended the learning opportunities by inviting students to revise and re-submit   their  work   after   getting  peers’   and/or  teacher’s   feedback.   Figure   5.7   shows   the   work   resubmitted   by   Group   3   after  receiving feedback.

Figure 5.7 Data collection template resubmitted by Group 3

141

5.3.3 Learning designs to foster skills for transforming information

As reported in Chapter Two, transforming information is found to be one of the most challenging CIL aspects for Hong Kong students. Activities that require students to synthesize, summarize, compare and contrast information from multiple sources have been found to be conducive to fostering skills in information transformation.

The following is one such curriculum example observed in a P.5 General Studies classroom. The topic for the learning unit was Our Community, and was organized in the form of a group project. The task was to plan a one-day field trip for a group of Primary 5 exchange students from Mainland China to visit their local region. The students were taught how to use Google Maps to draw the route and mark interesting sights for the trip, which may include historical attractions, landmarks, natural ecological attractions, local foods and food outlets, restaurants, religious  buildings  such  as  temples,  and  locations  for  the  region’s  unique  cultural  events or practices.

After completing the design for the field trip, students had to create a poster to publicize it and to invite registration. Figure 5.8 is the poster and field trip route map produced by one of the student groups, and Table 5.5 presents the rubric used for their assessment.

Table 5.5 Assessment criteria for the poster and route map

Checklist for assessing the poster

� Provides appropriate sightseeing locations � Includes time allocation in trip planning � At least 10 % of the text are written in students’ own words � Cites sources of information where relevant

Assessment criteria for the route map (Synthesizing information)

Novice ☐ Basic ☐ Proficient ☐ Advanced ☐

Not able to use Google Maps to locate the route

Able to use Google Maps to locate the route

Able to use Google Maps to locate the route and include some of the landscape photos

Able to use Google Maps to locate the route and include all of the landscape photos

142

The poster in Figure 5.8 demonstrates that the students were able to:

x Choose images relevant for the poster;

x Include relevant information in the poster;

x Maximize the use of the full page to create the poster; and

x Find information from various sources and use their own words to

summarize the information.

The  route  map  in  Figure  5.8  demonstrates  the  students’  ability  to:

x Integrate information to plan a route for the trip;

x Using Google map to create visually a route on a map; and

x Overlay the map on a landscape photo.

5.3.4 Designs to foster skills for sharing information

Communications technologies such as emails and discussion forums, and social

networking tools such as Facebook, Twitter and WhatsApp have been adopted

pervasively to mediate all kinds of formal and informal communication for

different purposes, including work, socializing and learning. Sharing information

is a major function of these communication activities.

In this section, we will introduce some examples of how collaborative tools

have  been  used  in  Hong  Kong  classrooms  to  support  students’ information sharing

for collaborative learning purposes. Figure 5.9 shows an image of the Google

calendar used by a group of students from several schools in planning their work

schedule to prepare for a creative drama competition.

Learning Management Systems (LMS) usually also provide support for

sharing and collaboration among learners. Figure 5.10 also indicates that the use of

LMS facilitated   students’   information   sharing.   In   this   example   all   the   students’  assignments in Chinese composition were uploaded to the platform for further

sharing  among  students.  Students  can  read  others’  work,  make  comments  as  well  as select good ones to save in their personal collection for further review.

143

Figu

re 5

.8 A

sam

ple p

oste

r and

fiel

d tr

ip ro

ute o

n G

oogl

e map

pro

duce

d by

a g

roup

of P

.5 st

uden

ts

144

Figure 5.9 Google calendar used by some students to plan their work schedule

Figure 5.10 An example of a Learning Management System customized to support shared reading and peer commenting of essays

145

Discussion forums are also frequently used to facilitate sharing of information and peer review. Figure 5.11 shows a screen-capture of one such discussion forum in which a group of students commented on the lyrics of a song written by one of their classmates. Table 5.6 shows one rubric that can be used for the assessment of a peer review type of forum postings.

Figure 5.11 A discussion forum where students commented on the lyrics written by their classmates Table 5.6 An assessment rubric for peer review type of forum postings

Novice Basic Proficient Advanced

Not able to post ideas on the forum.

Able to post some ideas on the forum.

Able to post ideas and give meaningful feedback to 1-2 peers.

Able to post ideas and give meaningful feedback to >2 peers.

146

5.4 An Extended Learning Design to Foster Learning in Multiple CIL Aspects

The size of a learning unit often limits the complexity of the learning tasks that can be presented to students. Longer learning units that span days or weeks involving tasks that need to be conducted both in school and at home often provide more opportunities for students to develop higher level CIL outcomes. These often require students to exercise skills in multiple CIL aspects in a meaningful and holistic fashion. In this section, we describe one such learning design that was collaboratively planned and jointly developed by Chinese Language teachers in two primary schools in one of the e-Learning Pilot Scheme projects (2011-14) in Hong Kong.

The duration of this learning unit was one month and the goal was for the students to write an essay or a prose on one shop of their choice in the Western District, which sells traditional products that have become relatively less popular in Hong Kong, such as paper craft shops (紙扎舖 ). The teachers wanted the students to have some experiential understanding, and preferably also some affective connections with the topic that they were to write on. To achieve this, the teachers designed a variety of activities culminating in a field trip to a shop of the students’  choosing  to  collect  first  hand  information  about  the  topic  they  were  to  write about.

The entire learning unit comprised five stages. In Stage One, the teacher introduced the Chinese prose genre 抒情文 giving several sample articles related to Hong Kong cultural artefacts and practices. A discussion forum was also created on the school LMS for students to post and share related articles and pictures. Students then had to decide individually which kind of traditional shop they would like to write about, and search for information about that kind of shop. Sample postings made by students on the discussion forum in this Stage shown in Figure 5.12 demonstrate their ability to use keywords to access information, as well as using the forum to manage and share information.

In Stage Two, students with similar interests were grouped to work together and decide on which shops to visit, plan the route for the visit and construct questions for the interview they were to conduct during the field trip. Students were asked to post and share their field trip plan on the forum. Teachers and students then read and gave suggestions and comments on plans posted by students from both schools. Figure 5.13 shows the target shop, route plan and interview questions prepared by one group of students in Stage 2. It shows that the students were able to use search engines to access information and decide on the specific shop to visit, manage route information using Google Maps, and transform information in designing the interview questions using their own words. Each group further planned detailed work allocation to ensure that the plan could be carried out smoothly during the field trip.

147

Figu

re 5

.12

Post

ings

of i

nfor

mat

ion

on th

e dis

cuss

ion

foru

m in

Stage  1,  d

emon

strating  students’  ability  to  use  keywords  to  

acce

ss in

form

atio

n, u

se th

e for

um to

man

age a

nd sh

are i

nfor

mat

ion

148

Figu

re 5

.13

The t

arge

t sho

p, ro

ute p

lan

and

inte

rvie

w q

uest

ions

pre

pare

d by

one

gro

up o

f stu

dent

s in

Sta

ge 2

, de

mon

stra

ting

thei

r abi

litie

s to

acce

ss, m

anag

e and

tran

sfor

m in

form

atio

n

149

Stage Three was the field trip. Students in the two schools met and travelled together to the Western District to visit the shop according to their plan and to interview the owner of the shop. Parents participated in the field trip as volunteers to assist the teachers in looking after the students.

During the field trip, students used various technologies such as digital cameras and oral dictation Apps on their mobile phones to record data. They then used GoogleDoc to enter and organize their collected data.

Figure 5.14 shows a photo of students conducting an interview with the shop-owner, the postings of information collected and their reflections on the field trip experience.  The  students’  work  during  this  stage  shows  their  ability  to  manage  and  share information as well as evaluate information.

In Stage Four, back in the classroom, students were asked to work with their group mates to construct an outline of their essay about the shop using a mind-map. After this, each student then write their own essay on the shop and post that to the LMS. Both tasks requires students to transform the information gathered and to create their own group and individual artefacts. Figures 5.15 and 5.16 show samples of the artefacts created by students during this stage

Finally, in Stage Five, students in the two schools were given opportunities to undertake self- and peer- evaluation of the essays uploaded to the LMS. They could read as many  of   their  peers’  uploaded   essays   as   they  wished.   In   reading  peers’  essays,  students  could  post  comments,  click  “like”  to  indicate  approval,  and  also   choose   to   “collect”   particularly   good   ones   to   put   into   one’s   own   private  collection for future review. On the LMS, both teachers and students could see, for each posted essay, the number of times the essay has been viewed, liked, and collected by other students.

Figure 5.17 shows the rubrics for self- and peer- assessment of the essays, and the peer feedback given by students to their peers online. These artefacts demonstrate  students’  ability  to  evaluate  and  share  information.

150

Figu

re 5

.14

In S

tage

3, s

tude

nts c

ondu

cted

the i

nter

view

dur

ing

the f

ield

trip

, the

n po

sted

the c

olle

cted

info

rmat

ion

and

thei

r ow

n re

flect

ions

onl

ine a

fter t

he v

isit

151

Figu

re 5

.15

Art

efac

ts c

reat

ed b

y st

uden

ts in

Sta

ge F

our:

The g

roup

min

d m

ap a

nd a

3D

pap

er m

odel

cre

ated

by

a st

uden

t who

com

plet

ed

his e

ssay

wri

ting early.  These  artefacts  dem

onstrate  students’  ability  to  tran

sform  and  create  info

rmat

ion

152

Figu

re 5

.16

An

essa

y w

ritt

en b

y on

e of t

he st

uden

ts, d

emon

stra

ting

his a

bilit

y to

tran

sfor

m a

nd c

reat

e in

form

atio

n

153

Figu

re 5

.17

The r

ubri

c fo

r sel

f- an

d pe

er- a

sses

smen

t of t

he es

says

, and

the p

eer f

eedb

ack

give

n by

stu

dent

s to

thei

r pe

ers o

nlin

e

154

5.5 Summary

In this chapter, we have introduced a number of learning designs and assessment instruments collected from the classrooms and schools in Hong Kong in which teachers  aimed  to  foster  students’  information  literacy  skills,  covering  all  six  CIL  aspects (i.e. all except knowing about and understanding computer use). Actually, even this last aspect is implicitly included in the learning tasks but was not explicitly assessed.

In all these cases, a crucial feature of the learning tasks is the openness of the tasks, which provided students with the opportunities to explore and inquire. Another feature is that the technologies used are not highly sophisticated. In fact these were relatively general, but would need the availability of an LMS and other social networking software to support collaboration, communication and sharing of information.

Another important observation is the advantage of an extended learning design, which gives students rich experiences in learning subject matter knowledge and information literacy in context. Such authentic learning contexts generally stimulate intrinsic motivation in learners, enhancing the effectiveness of the learning.

A third observation is that it is desirable for students to be provided with the same technology platform for learning in different subject matter contexts and tasks. It takes time for both teachers and students to get accustomed to the interfaces and functionalities of a new technology, creating additional cognitive and organizational burdens for all involved. This additional effort would be minimized  if  the  same  technology  were  used  throughout  the  course  of  a  student’s  learning.

155

References

ACARA (Australian Curriculum, Assessment and Reporting Authority) (2015). National Assessment Program – ICT Literacy Years 6 & 10 Assessment

Framework 2014. Sydney: ACARA. Retrieved August 25, 2015, from http://www.nap.edu.au/verve/_resources/NAP-ICT_Assessment_Framework_2014.pdf

American Library Association. (1989). Presidential Committee on Information

Literacy: Final Report. : American Library Association. Arter, J., & McTighe, J. (2001). Scoring Rubrics in the Classroom: Using Performance

Criteria for Assessing and Improving Student Performance. Thousand Oaks, California: Corwin Press Inc.

Bransford, J. D., Brown, A. L., & Cocking, R. R. (1999). How people learn: Brain,

mind, experience, and school (expanded edition). Washington, DC: National Academies Press.

Bray, M., & Lykins, C. (2012). Shadow Education Private Supplementary Tutoring and

Its Implications for Policy Makers in Asia. Mandaluyong City, Philippines: Asian Development Bank.

Bundy, A. (2004). Australian and New Zealand information literacy framework:

principles, standards and practice (2nd ed.). Adelaide: Australian and New Zeal and Institute for Information Literacy (ANZIIL).

Dann, R. (2002). Promoting assessment as learning: Improving the learning process.

London: Routledge/Falmer. European Commission (2000). Communication from the commission to the council and

the European Parliament: The elearning action plan - Designing  tomorrow’s  

education. Brussels: Commission of the European Communities. Education and Manpower Bureau (EMB). (2005). A study on the development of an

information literacy framework for Hong Kong students. Hong Kong: Government printings.

156

Education and Manpower Bureau (EMB) (1998). Information technology for learning

in a new era: five-year strategy 1998/99 to 2002/03. Hong Kong: Education

and Manpower Bureau, Hong Kong SAR Government. Retrieved August

17, 2009, from

http://www.edb.gov.hk/index.aspx?langno=1&nodeID=425.

Education and Manpower Bureau (EMB), (2000) Information Learning target: A

guideline for schools to organize teaching and learning activities to develop our

students’  capacity  in  using  IT. [Electronic Version]. Retrieved September 11,

2015, from http://www.edb.gov.hk/attachment/en/curriculum-

development/4-key-tasks/it-for-interactive-learning/it-learning-

targets/ITLT-e.pdf

Education and Manpower Bureau (EMB) (2004). Empowering learning and teaching

with information technology. Hong Kong: Education and Manpower

Bureau, Hong Kong SAR Government. Retrieved August 17, 2009, from

http://www.edb.gov.hk/index.aspx?langno=1&nodeID=2497.

Education Bureau (EDB) (2008). Right technology at the right time for the right task.

Hong Kong: Education Bureau (EDB), Hong Kong SAR Government.

Retrieved August 17, 2015, from http://www.edb.gov.hk/en/edu-

system/primary-secondary/applicable-to-primary-secondary/it-in-

edu/consultation-3rd-ited.html

Education Bureau (EDB) (2006). The Learning Outcomes Framework (LOF) in the

Science Key Learning Area. Hong Kong: Government printings. Retrieved

August 17, 2009, from

http://cd1.edb.hkedcity.net/cd/science/lof_e/index.htm

Educational Testing Service (ETS) (2003). Succeeding in the 21st century: What

higher education must do to address the gap in information and communication

technology proficiencies. Princeton: NJ: Author.

Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E. (2014). Preparing

for life in a digital age: The IEA International Computer and Information Literacy

Study International Report: Springer Open.

Fraillon, J., Schulz, W., & Ainley, J. (2013). International Computer and Information

Literacy Study: Assessment framework. Amsterdam: IEA.

157

Fraillon, J., Schulz, W., Friedman, T., Ainley, J., & Gebhardt, E. (2015). International Computer and Information Literacy Study 2013 technical report. Amsterdam, the Netherlands: International Association for the Evaluation of Educational Achievement (IEA).

Goodrich, H. (1997). Understanding rubrics. Educational Leadership, 54(4), 14–17. Helvoort, J. v. (2010). A scoring rubric for performance assessment of information literacy in Dutch Higher Education. Journal of Information Literacy, 4(1), 22-39. Herman, J., Aschbacher, P., & Winters, L. (1992). A practical guide to alternative

assessment. Alexandria, VA: Association for Supervision and Curriculum Development.

International Society for Technology for Education (ISTE) (1998). National

educational standards for students (NETS) 1998 [Electronic Version]. Retrieved April 11, 2003, from http://www.iste.org/Content/NavigationMenu/NETS/ForStudents/1998Standards/ NETS_for_Students_1998_Standards.pdf

International Society for Technology for Education. (2007). National Educational Technology Standards (NETS.S) and Performance Indicators for Students. Retrieved August 25, 2015, from http://smsd.us/webpages/swilliams/files/NETS_for_Students_2007_Standards1.pdf

KOSIS (Korean Statistical Information Service) (2014). After-school programs participation rate, Statistical Database. Retrieved August 28, 2015, from http://kosis.kr/eng/statisticsList/statisticsList_01List.jsp?vwcd=MT_ETITLE&parentId=C#SubCont

Knight, L. A. (2006). Using rubrics to assess information literacy. Reference Services Review, 34 (1), 43-55.

Law, N., Lee, Y., & Chow, A. (2002). Practice characteristics that lead to "21st

century learning outcomes". Journal of Computer Assisted Learning, 18(4), 415-426.

Law, N., Lee, Y., & Yuen, A. (2010). Assessing the effects of ICT in education. In

Friedrich Scheuermann and Francesc Pedró. (ed) Assessing the effects of ICT in education: indicators, criteria and benchmarks for international comparisons (pp.143-164) Luxembourg: European Commission

158

Law, N., Yuen, A., Shum, M., & Lee, Y. ( 2007). Phase (II) Study on Evaluating the Effectiveness  of  the  ‘Empowering Learning and Teaching with Information Technology’  strategies  (2004/2007)  Final  Report. [Electronic Version]. Retrieved July, 2015, from http://www.edb.gov.hk/en/edu-system/primary-secondary/applicable-to-primary-secondary/it-in-edu/phase-II-study-on-2nd-ited.html

Moskal, B. M. (2000). Scoring Rubrics: What, when, and how? [Electronic

Version]. Practical Assessment, Research, and Evaluation, 7(3). Rasch, G. (1960). Probabilistic models for some intelligence and achievement tests.

Copenhagen, Denmark: Danish Institute for Educational Research. SCONUL. (1999). Information Skills in Higher Education: A SCONUL Position Paper

[Electronic Version]. Retrieved March 24, 2003, from http://www.sconul.ac.uk/activities/inf_lit/papers/Seven_pillars.html.

SCONUL (2004), Information Support for eLearning: Principles and Practice, Rev. ed.,

Society of College, National and University Libraries, London, [Electronic Version]. Retrieved June 24, 2007, from http://www.sconul.ac.uk/publications/pubs/info_support_elearning.pdf

SRI. (2011). Innovative Teaching and Learning Research 2011 Findings and

Implications. Stanford, CA: SRI. Stenmark, J. (1991). Mathematics Assessment: Myths, Models, Good Questions, and

Practical Suggestions. Reston, VA: NCTM. Stergar, C. (2005). Performance tasks, checklists, and rubrics. Glenview, IL:

LessonLab. Von Davier, M., Gonzalez, E., & Mislevy, R. (2009). What are plausible values and

why are they useful. IERI monograph series, 2, 9-36.

8659607898819

ISBN 978988186596090000 >

People in Hong Kong generally assume that students in our schools now have fairly well-developed computer skills since they start playing with computer technology almost from birth, and are often referred to as “digital natives”. However, do these use experiences with digital devices mean that our students already have the competence to engage in life-long learning to tackle problems using ICT as a core 21st century skill? At the same time, schools in Hong Kong have experienced the launch of four IT in education strategies since 1998. Are teachers prepared to implement e-learning pedagogy, and are principals prepared to lead e-learning innovations in schools to foster the development of computer and information literacy (CIL) in their students? This book aims to identify the strengths and weaknesses of Hong Kong students’ Computer and Information Literacy (CIL) in comparison with their international counterparts through in-depth analysis of the data collected in the ICILS 2013 Study. The final chapter provides curriculum exemplars to illustrate the key characteristics of pedagogical and assessment designs that are conducive to enhancing students’ CIL. These exemplars were generated in primary and secondary schools in Hong Kong from past and current CITE projects. We hope that this book will be a useful resource for primary and secondary school teachers, principals, education policy-makers, teacher educators and members of the community who are interested in helping students learn how to use ICT tools productively for lifelong learning in the 21st century.