KNOWLEDGE BUILDING IN CONTINUING MEDICAL ...

KNOWLEDGE BUILDING IN CONTINUING MEDICAL EDUCATION

by

Leila Rachel Lax

A thesis submitted in conformity with the requirements

for the degree of Doctor of Philosophy

Department of Theory and Policy Studies in Education

Ontario Institute for Studies in Education

University of Toronto

© Copyright by Leila Rachel Lax 2012

ii

KNOWLEDGE BUILDING IN CONTINUING MEDICAL EDUCATION

Doctor of Philosophy, 2012

Leila Rachel Lax

Department of Theory and Policy Studies in Education

University of Toronto

ABSTRACT

Continuing medical education has been characterized as didactic and ineffective. This thesis

explores the use of Knowledge Building theory, pedagogy, and technology to test an alternative

model for physician engagement—one that emphasizes sustained and creative work with ideas.

Several important conceptual changes in continuing medical education are implied by the

Knowledge Building model—changes that extend the traditional approach through engagement

in (a) collective responsibility for group achievements rather than exclusive focus on individual

advancement and (b) work in design-mode, with ideas treated as objects of creation and

assemblage into larger wholes and new applications, with extension beyond belief-mode where

evidence-based acceptance or rejection of beliefs dominates. The goal is to engage physicians in

“cultures of participation” where individual learning and collective knowledge invention or

metadesign advance in parallel.

This study was conducted in a continuing medical education End-of-Life Care Distance

Education course, for family physicians, from 2004 to 2009. A mixed methods case study

methodology was used to determine if social-mediated Knowledge Building improved

physicians’ knowledge, and if so, what social network structural relationships and sociocognitive

dynamics support knowledge improvement, democratization of knowledge, and a metadesign

perspective.

iii

Traditional pre-/posttest learning measures across 4-years showed significant gains (9% on

paired t-test = 5.34, p < 0.001) and large effect size (0.82). Social network analysis of ten

2008/2009 modules showed significant difference in density of build-on notes across groups.

Additional results demonstrated a relationship between high knowledge gains and social network

measures of centrality/distribution and cohesion. Correlation of posttest scores with centrality

variables were all positive. Position/power analyses highlighted core-periphery sociocognitive

dynamics between the facilitator and students. Facilitators most often evoked partner/expert

relationships. Questions rather than statements dominated the discourse; discourse complexity

was elaborated/compiled as opposed to reduced/dispersed. Themes beyond predefined learning

objectives emerged and Knowledge Building principles of community responsibility, idea

improvability, and democratization of knowledge were evident. Overall, results demonstrate the

potential of collective Knowledge Building and design-mode work in continuing medical

education, with individual learning representing an important by-product. There were no

discernible decrements in performance, suggesting significant advantages rather than tradeoffs

from engagement in Knowledge Building.

iv

ACKNOWLEDGEMENTS

I would like to express my sincere gratitude to my thesis supervisor and mentor, Dr. Marlene

Scardamalia, who has forever changed the way I think, work, and look at the world, and inspired

my desire to “advance the edge.” I am also most grateful to the members of my thesis committee,

Dr. Michael Skolnik, Dr. Jim Hewitt, and Dr. Judy Watt-Watson, who have been insightful

teachers and wonderful mentors throughout my years at OISE, and my external appraiser, Dr.

Beatrice Ligorio. I would also like to acknowledge my fellow researchers at the Institute of

Knowledge Innovation and Technology at OISE/UT and the international community of the

Knowledge Society Network. I extend sincere thanks to my external social network analyst, Dr.

Don Philip, statistician Ms. Susan Elgie, and editors Susana Larosa and Mary Anne Carswell. In

addition, I thank the many academic/clinicians, professors, educational researchers, and

simulators I have had the opportunity to work with throughout the years, at the Wilson Centre for

Research in Medical Education, the Faculty of Medicine, the University of Toronto Centre for

the Study of Pain, the Faculty of Nursing, and the Standardized Patient Program, who have

become valued colleagues throughout the years, especially Dr. Judy Watt-Watson, Dr. Michael

McGillion, Dr. Lynn Russell, Dr. Cathy Smith, Laura Jayne Nelles, Dr. Niall Byrne, and Dr.

Glenn Regehr. The End-of-Life Care Distance Education Program could not have been

developed without the vision of Dr. Lawrence Librach. I thank my extraordinary collaborator,

Dr. Anita Singh, for her medical expertise in palliative care and contributions to the creation,

implementation, and ongoing success of the End-of-Life Care Distance Education Program. I am

indebted to the palliative care specialists, Dr. Anoo Tamber, Dr. Hyon Kim, and Dr. Paolo

Mazzotta for their dedication to facilitating online Knowledge Building in this program and to

Ms. Nancy Bush, for her excellent administrative and organizational skills. I gratefully

acknowledge the former directors of the Faculty of Medicine Discovery Common, Dr. Lawrence

Spero and Dr. Avi Hyman, and media specialists Prof. Meaghan Brierley, Jenn Tse, and Ju Ho

Park, for their support and contribution to media development. And last but certainly not least, I

sincerely thank my colleagues in Biomedical Communications, especially Dr. Linda Wilson-

Pauwels, Prof. Nicholas Woolridge, Prof. Emerita Margot Mackay, and Prof. Emerita Nancy

Joy, and my students whose dedication to improvable ideas has been a constant inspiration.

v

TABLE OF CONTENTS

Abstract ........................................................................................................................................... ii

Acknowledgements ........................................................................................................................ iv

List of Tables ................................................................................................................................. xi

List of Figures .............................................................................................................................. xiv

Dedication .................................................................................................................................. xviii

CHAPTER 1: INTRODUCTION ....................................................................................................1

1.1 Overview ..............................................................................................................................1

1.2 Context .................................................................................................................................2

1.3 Theoretical Foundations.......................................................................................................4

1.4 Background ..........................................................................................................................7

1.5 Purpose ...............................................................................................................................10

1.6 Significance........................................................................................................................11

1.7 Research Questions ............................................................................................................12

1.7.1 Research Clusters and Subquestions ......................................................................12

1.8 Dissertation Organization ..................................................................................................14

CHAPTER 2: LITERATURE REVIEW .......................................................................................17

2.1 Introduction ........................................................................................................................17

2.2 Cluster 1: Traditional Conceptualizations of Continuing Medical Education ...................18

2.2.1 Continuing Medical Education for Individual Lifelong Learning .........................18

2.2.1.1 Definition ......................................................................................................18

2.2.1.2 From individual to socially mediated participatory metadesign ...................20

2.3 Cluster 2: Performance Over and Above Traditional Individual Outcomes, Through

Collective Knowledge Building .........................................................................................23

2.3.1 Knowledge Building Research in Medical and Health Sciences Education ..........23

2.3.2 Distinction Between Learning and Knowledge Building ......................................25

2.3.3 Historical and Cultural Relevance of Knowledge Building ..................................27

2.3.4 Belief Mode and Design Mode ..............................................................................31

2.3.5 The Knowledge Building Problem Space ..............................................................32

2.3.6. Participant Structures and the Codesign of Knowledge Building .........................33

2.3.6.1 Facilitating Knowledge Building ..................................................................33

2.3.6.2 Facilitating knowledge building in relationship to coaching

reflection-in-action ........................................................................................34

2.3.6.3 Expertise and social networks for cocreation ...............................................37

2.3.7 Knowledge Forum Suite of Analytic and Social Network Tools

to Support Knowledge Building ............................................................................38

2.4 Cluster 3: Social Network Measures and Sociocognitive Dynamics

That Enable Work Over and Above Traditional Learning ................................................40

2.4.1 Social Network Analysis in the Social Sciences....................................................40

2.4.2 Social Network Analysis and Power Structures in Education ...............................42

2.4.2.1 Power structures concepts .............................................................................42

2.4.2.2 Social network analysis of roles and power structures .................................44

2.4.2.3 Social network position and identity.............................................................44

2.4.2.4 Social network analysis of facilitator roles and participant structures .........45

2.4.3 Social Network Analysis of Sociocognitive Dynamics in

Knowledge Building ..............................................................................................46

vi

2.4.3.1 Social network analysis of Knowledge Building in K–12 classrooms .........46

2.4.3.2 Social network analysis of knowledge building in higher education ...........47

2.4.3.3 Social network analysis of an international knowledge building network ...48

2.4.4 Social Network Analysis in Medicine and Medical Education .............................49

2.4.5 Complexity of Social Network Discourse: Visual Representation,

Elaboration, and Identification of Misconceptions ................................................50

2.4.6 Analogies and Emergent Ideas in Abductive and Adductive Processes ................55

2.5 Summary and Implications ................................................................................................57

CHAPTER 3: METHODS .............................................................................................................59

3.1 Introduction ........................................................................................................................59

3.2 Context and Program Overview.........................................................................................59

3.3 Research Goal and Question ..............................................................................................60

3.4 Case Study Methodology ...................................................................................................61

3.5 Pedagogic Design...............................................................................................................62

3.5.1 Multimedia Design of Knowledge Forum Communal Space ................................63

3.5.2 Knowledge Building Theory-Based Design ..........................................................64

3.6 Research Methods ..............................................................................................................66

3.6.1 Participants .............................................................................................................67

3.6.2 Instrumentation and Procedures .............................................................................67

3.6.2.1 Pain knowledge pre- and posttests ................................................................68

3.6.2.2 Attitudes and opinions survey .......................................................................69

3.6.2.3 Knowledge Forum online activity and interactivity .....................................70

3.6.3 Data Collection and Analyses: Pilot Study 2004/2005 ..........................................70

3.6.4 Data Collection and Analyses Cluster 1: Traditional Measures

Across Years 2005/2009 ........................................................................................70

3.6.4.1 Analyses of pain knowledge improvement ...................................................70

3.6.4.2 Attitudes and opinions of collaborative online work ....................................70

3.6.5 Data Collection and Analyses Cluster 2: Activity, Interactivity,

and Social Network Measures Over and Above Traditional Measures .................71

3.6.5.1 Knowledge Forum online activity and interactivity .....................................71

3.6.5.2 Knowledge Forum graphic contribution and social network analyses .........71

3.6.5.3 2-way ANOVA .............................................................................................71

3.6.5.4 Social network structural analyses ................................................................71

3.6.5.5 Significant difference and effect size of social network density and

centrality measures between groups, with and without facilitator ................72

3.6.5.6 Relationship of social network structural analyses of the three

pain modules to pain pre-/posttest scores .....................................................72

3.6.6. Data Collection and Analyses Cluster 3: Social Network Analyses and

Measures of Sociocognitive Dynamics That Support Knowledge Building

Outcomes, Over and Above Learning ...................................................................72

3.6.6.1 Social network position and power analyses ................................................72

3.6.6.2 Social network content analyses ...................................................................73

3.6.6.2.1 Social network analysis of facilitator and student patterns

of discourse ..........................................................................................73

3.6.6.2.2 Content analysis of themes of discourse, beyond learning

objectives .............................................................................................73

3.6.6.2.3 Complexity of clinical discourse .........................................................74

vii

3.6.6.2.4 Evidence of Knowledge Building indicators and exemplars ...............74

3.6.7 Summary ................................................................................................................74

CHAPTER 4: RESULTS—PILOT STUDY .................................................................................78

4.1 Introduction ........................................................................................................................78

4.2 Pilot Study, 2004/2005: Results .........................................................................................78

4.2.1 Pain Knowledge, 2004/2005: Pre- and Posttest Results ........................................78

4.2.2 Pilot Study, 2004/2005: Attitudes and Opinions Survey Results ..........................80

4.2.3 Pilot Study, 2004/2005: Online Activity Analytic Toolkit Measures....................82

4.2.4 Pilot Study, 2004/2005: Summary and Iterative Design Recommendations .........83

CHAPTER 5: RESULTS—CLUSTER 1: TRADITIONAL OUTCOME MEASURES ..............85

5.1 Introduction ........................................................................................................................85

5.2 Pain Knowledge Pre-/Posttest Results by Year and Cumulatively Across Four Years .....86

5.2.1 Pain Knowledge 2005/2006 Pre-/Posttest Matched Results ..................................87




5.2.5 Cumulative Matched Results of 2005–2009 Pain Knowledge Pre-/Posttests .......90

5.2.6 Summary of Results of Pain Knowledge Pre-/Posttests ........................................90

5.3 Attitude and Opinion Survey Results 2005–2009 .............................................................91

5.4 Summary of Traditional Outcome Measures: 2005–2009 Results of Pain

Knowledge Pre-/Posttest Results and Attitude and Opinion Survey Results ....................96

CHAPTER 6: RESULTS—CLUSTER 2: PERFORMANCE OVER AND ABOVE

TRADITIONAL MEASURES .................................................................................................97

6.1 Introduction ........................................................................................................................97

6.2 Online Activity and Interactivity Measures Results ..........................................................98

6.2.1 Results of 2005–2009 Read, Write, Build-On Measures .......................................99

6.3 Results of 2008/2009 Online Performance and Social Network Relationship

Measures (Beyond Learning) ...........................................................................................102

6.3.1 Group 1, 2008/2009 ATK and Social Network Assessment Measures ...............103

6.3.1.1 Group 1, Mr. Singh’s Pain, Part 1 ...............................................................103


6.3.1.3 Group 1, Mary’s Misery, Pain Module 3 ....................................................109

6.3.1.4 Group 1, Judy’s Last Days, Part 1 ..............................................................112

6.3.1.5 Group 1, Judy’s Last Days, Part 2 ..............................................................115

6.3.2 Group 2, 2008/2009 ATK and Social Network Assessment Measures ...............118



6.3.2.3 Group 2, Mary’s Misery, Pain Module 3, Group 2 .....................................124

6.3.2.4 Judy’s Last Days, Part 2, Group 2 ..............................................................127

6.3.2.5 Judy’s Last Days, Part 2, Group 2 ..............................................................130

6.3.3 Social Network Pattern Analysis Across Modules and Comparatively

Between Groups, 2008/2009 ................................................................................133

6.3.4 Summary of Social Network Analysis and Knowledge Forum Tools .................138

6.4 Results for Groups 1 and 2 of 2-Way ANOVA of 2008/2009 Pain Knowledge

Pre-/Posttests ....................................................................................................................139

viii

6.5 Social Network Structural Analysis Results 2008/2009 ..................................................141

6.5.1 Structural SNA: Network Links and Network Density Results...........................142

6.5.1.1 Network density results of t-tests and effect ...............................................144

6.5.2 Social Network: Eigenvector, In-degree, and Out-degree Centrality Results .....145

6.5.2.1 t-test and effect size results of social network centrality ............................147

6.5.2.2 t-test and effect size results of social network density and centrality

measures across groups ...............................................................................149

6.6 Social Network Analyses Results of Cliques (K) and Cohesion Index of Build-On

Ideas Only ........................................................................................................................150

6.7 Relationship of Social Networks to Knowledge Improvement Scores ............................153

6.7.1 Relationship of Social Network Structural Analyses of the Three Pain

Modules to Pain Knowledge Improvement Pre-/Posttest Scores ........................153

6.7.2 Correlation Between Results of Social Network Analyses and Pain

Posttest Scores .....................................................................................................155

6.8 Cluster 2 Summary of Performance Measures (Over and Above

Traditional Learning) From Social Network Structural Results ......................................156

CHAPTER 7: RESULTS—CLUSTER 3: SOCIAL NETWORK POSITION/POWER

ANALYSIS AND SOCIOCOGNITIVE DYNAMICS OF KNOWLEDGE BUILDING .....159

7.1 Introduction ......................................................................................................................159

7.2 Results of Social Network Position/Power Analysis .......................................................161

7.2.1 Relationship of Individual Student Social Network Position Scores

to Difference on Pre-/Posttest Knowledge Scores ...............................................161

7.2.2 Social Network Position, and Power Maps and Measures ..................................162

7.2.2.1 Position/power results for Group 1, Mr. Singh’s Pain, Part 1 ....................163


7.2.2.3 Position/power results for Group 1, Mary’s Misery ...................................167

7.2.2.4 Position/power results for Group 1, Judy’s Last Days, Part 1 ....................169


7.2.2.6 Position/power analysis interpretation and conclusions for Group 1 .........174



7.2.2.9 Position/power results for Group 2, Mary’s Misery ...................................178



7.2.2.12 Position/power analysis interpretation and conclusion for Group 2 ...........185

7.3 Results of Social Network Content Analyses ..................................................................186

7.3.1 Social Network Analysis of Facilitator/Students Patterns of Discourse ..............187

7.3.1.1 Patterns of facilitator discourse ...................................................................187

7.3.1.2 Patterns of discourse statements and questions ..........................................188

7.3.2 Content Analysis of Themes, Beyond the Predetermined

Learning Objectives .............................................................................................191

7.3.2.1 Content analysis of knowledge work ..........................................................191

7.3.2.2 Relationship of themes, threads, and learning objectives in the social

network discourse .......................................................................................192

7.3.2.3 Summary of emergent themes and metadesign results ...............................195

7.3.3 Social Network Analysis of Complexity of Discourse ........................................196

ix

7.3.4 Evidence of Knowledge Building Indicators Within Social

Network Discourse...............................................................................................198

7.3.5 Cluster 3 Summary of Social Network Position and Power Analysis,

Sociocognitive Dynamics, and Indicators of Knowledge Building .....................210

7.3.5.1.1 Facilitation patterns ............................................................................211

7.3.5.1.2 Belief-mode and design-mode knowledge work and its

relationship to predefined learning objectives ...................................211

7.3.5.1.3 Emergent themes and metadesign (beyond predefined learning

objectives) ..........................................................................................211

7.3.5.1.4 Results of Knowledge Building indicators within social

network discourse ..............................................................................211

7.4 Overview of Results .........................................................................................................212

7.4.1 Pilot Study 2004/2005..........................................................................................212

7.4.2 Cluster 1: Traditional Measures ...........................................................................212

7.4.2.1 Matched results of pain knowledge pre-/posttests, 2005–2009 ..................212

7.4.2.2 Attitude and opinion results 2005–2009 .....................................................213

7.4.3 Cluster 2: Performance Over and Above Traditional Measures ..........................213

7.4.3.1 ATK Measures 2005–2009 .........................................................................213

7.4.3.2 Social network results (KF analytic tools) 2008/2009: Groups 1 and 2 .....214

7.4.3.2.1 Mr. Singh’s Pain, Part 1: Group 1 .....................................................214


7.4.3.2.3 Mary’s Misery: Group 1 ....................................................................214

7.4.3.2.4 Judy’s Last Days, Part 1: Group 1 .....................................................215




7.4.3.2.8 Mary’s Misery: Group 2 ....................................................................215



7.4.3.2.11 Results of 2-way ANOVAs on 2008–2009 Groups 1 and 2 ..............216

7.4.3.2.12 Network links and density measures and tests of significant

difference and effect size ...................................................................216

7.4.3.2.13 Social network measures of centrality: Eigenvector, in-degree,

and out-degree; and results of t-test and effect size ...........................217

7.4.3.2.14 Significant difference between groups in social network density

measures .............................................................................................217

7.4.3.2.15 Clique members and clique cohesion index.......................................217

7.4.3.2.16 Relationship of SN structural analysis of three pain modules to

pre-/posttest knowledge improvement scores ....................................218

7.4.4 Cluster 3: Social Network Analyses and Sociocognitive Dynamics That

Support Knowledge Building Over and Above Learning ....................................218

7.4.4.1 Results of social network position/power analysis 2008/2009 ...................218

7.4.4.2 Results of social network content analysis 2008/2009 ...............................218

7.4.4.2.1 Facilitator/students patterns of discourse patterns .............................219

7.4.4.2.2 Results of analysis of complexity of discourse ..................................219

7.4.4.2.3 Emergent themes/threads in knowledge work (beyond

predefined learning objectives) ..........................................................219

x

7.4.4.2.4 Results of Knowledge Building indicators within social network

discourse ............................................................................................219

CHAPTER 8: DISCUSSION .......................................................................................................221

8.1 Introduction ......................................................................................................................221

8.2 Summary of Research ......................................................................................................221

8.3 Significance of Cluster 1 Research Results: Traditional Learning Outcomes

and the Relationship to Continuing Medical Education, Traditionally Conceived .........225

8.4 Significance of Cluster 2 Research Results: Social Network Performance

Measures Over and Above Learning ...............................................................................226

8.5 Significance of Cluster 3 Research Results: Sociocognitive Dynamics

That Enable Work Over and Above Learning .................................................................229

8.5.1 Facilitator/Participant Sociocognitive Dynamics ................................................229

8.5.2 Belief- and Design-Mode Knowledge Work .......................................................230

8.5.3 Knowledge Building Principles and Metadesign Concepts .................................231

8.5.3 Collective Responsibility and Democratizing Knowledge ..................................232

8.6 Summary ..........................................................................................................................236

CHAPTER 9: CONCLUSIONS ..................................................................................................237

9.1 Introduction ......................................................................................................................237

9.2 Research Questions and Answers ....................................................................................237

9.3 Strengths of This Research Study ....................................................................................239

9.4 Limitations .......................................................................................................................240

9.5 Future Research ...............................................................................................................240

9.6 Final Remarks ..................................................................................................................241

REFERENCES ............................................................................................................................243

APPENDICES .............................................................................................................................263

Appendix A: Information Letter and Informed Consent Form ....................................................264

Appendix B: End-of-Life Care Distance Education Program Homepage and Schedule .............266

Appendix C: End-of-Life Care Distance Education Program Modules in

Knowledge Forum® and Multimedia Case Note ....................................................................267

Appendix D: Pain Pre-/Posttest Item Analysis 2004/2005 ..........................................................268

Appendix E: Pain Pre-/Posttest Item Analysis 2005/2006 ..........................................................272

xi

LIST OF TABLES

Table 1. Research Clusters and Organization of Dissertation .....................................................16

Table 2. End-of-Life Care Distance Education Program Participants and Data Collection

Instruments ................................................................................................................69

Table 3. Demographics of the Study Population .........................................................................75

Table 4. Research Question/Subquestions, Data Analysis, and Data Sources ............................75

Table 5. Matched Results From the EoL Care Pain Pre-/Posttests, 2004/2005 ...........................79

Table 6. Responses to Attitudes and Opinions Survey, 2004/2005 (n = 10 of 13) ......................81

Table 7. Online Activity Measures by Group, 2004/2005 ...........................................................82

Table 8. Research Question .........................................................................................................85

Table 9. Cluster 1: Traditional Measures .....................................................................................86

Table 10. Matched Results of Pain Pre-/Posttests, 2005/2006 ......................................................87

Table 11. Matched Results of Pain Pre-/Posttests, 2006/2007 ......................................................88

Table 12. Matched Results of the 2007/2008 Pain Pre-/Posttests .................................................89

Table 13. Matched Results of the 2008/2009 Pain Pre-/Posttests .................................................89

Table 14. Cumulative Matched Results of the 2005–2009 Pain Pre-/Posttests .............................90

Table 15. Percentaged Results of Attitude and Opinions Summative Survey, 2005–2009 ...........91

Table 16. Cluster 2: Beyond Traditional Measures .......................................................................97

Table 17. Online Activity Measures from 2005/2006, 2006/2007, and 2007/2008 ....................100

Table 18. Summary of Online Activity and Interactivity Measures, 2008/2009 .........................101

Table 19. Results of 2-way ANOVA Groups 1 and 2, 2008/2009 ..............................................140

Table 20. Number of Edges and Network Density of Build-on Notes With and Without

Facilitator, Group 1 .................................................................................................142

Table 21. Number of Edges (Links) and Network Density of Build-ons With and Without


Table 22. Summary of Edges/Links and Network Density: Groups 1 and 2 ...............................144

Table 23. Comparison of Density with and Without Facilitator of Notes

Built-On and Read ...................................................................................................145

Table 24. Social Network Centrality Measures (in Percentages) With and Without


Table 25. Social Network Centrality Measures (in Percentages) With and Without


Table 26. Comparisons of SN Centrality Measures With and Without Facilitator,

by Group, Across All Modules ................................................................................148

Table 27. Comparisons of SN Measures, between Groups 1 and 2, With Facilitator,

Across All Modules .................................................................................................149

xii

Table 28. Clique Members and Cohesion Index Results of Build-Ons by Group

With Facilitator ........................................................................................................151

Table 29. Clique and Cohesion Index Results of Build-On Notes in Groups1 and 2,

With Facilitators ......................................................................................................152

Table 30. Comparative Summary of SN Results of Groups 1 and 2 in Three Pain Modules......154

Table 31. Comparisons of Social Network Measures of Build-Ons, Between Groups,

Across Three Pain Modules .....................................................................................154

Table 32. Results of 2008/2009 Pain Pre-/Posttests ....................................................................155

Table 33. Spearman Correlations of the Post Score with Social Network Variables ..................156

Table 34. Cluster 3: Social Network Analysis of Sociocognitive Dynamics ..............................160

Table 35. Group 1 Individual Differences in Student Pain Knowledge Pre-/Posttest Scores .....161

Table 36. Group 2 Individual Differences in Student Pain Knowledge Pre-/Posttest Scores .....162

Table 37. Social Network Position/Power Map and Centrality Measures for Group 1,

Mr. Singh’s Pain, Part 1, Build-On Notes ...............................................................164




Mary’s Misery, Build-On Notes ..............................................................................168


Judy’s Last Days, Part 1, Build-On Notes ...............................................................170








Mary’s Misery, Build-On Notes ..............................................................................179





Table 47. Facilitator/Participant Discourse Patterns: Discourse Stance ......................................187

Table 48. Patterns of Discourse Statements and Questions by Facilitators and Students,

and Student/Facilitator Patterns at the Core ............................................................189

Table 49. Results of Knowledge Work With Predefined Learning Objectives or

Emergent Ideas ........................................................................................................191

Table 50. Summary of Emergent Themes/Threads Beyond Learning Objectives ......................194

xiii

Table 51. Semantic Analysis Results of Complexity of Discourse .............................................198

Table 52. Knowledge Building Principles Demonstrated in the Discourse.................................199

Table 53. Clusters Summary of Research Questions and Results of Analyses ...........................222

xiv

LIST OF FIGURES

Figure 1. Total note contributions Group 1, Mr. Singh’s Pain, Part 1, 2008/2009 ......................104

Figure 2. Total number of build-on notes created Group 1, Mr. Singh’s Pain, Part 1,

2008/2009 ................................................................................................................104

Figure 3. Social network analysis of who built-on whose notes, Group 1, Mr. Singh’s Pain,

Part 1, 2008/2009 .....................................................................................................105

Figure 4. Total number of notes read, Group 1, Mr. Singh’s Pain, Part 1, 2008/2009 ................105

Figure 5. Social network analysis of who read whose notes, Group 1, Mr. Singh’s Pain,

Part 1, 2008/2009 .....................................................................................................106

Figure 6. Total note contributions, Group 1, Mr. Singh’s Pain, Part 2, 2008/2009 .....................107


2008/2009 ................................................................................................................107

Figure 8. Social network analysis of who built-on whose notes, Group 1, Mr. Singh’s Pain,

Part 2, 2008/2009 .....................................................................................................108

Figure 9. Total number of notes read, Group 1, Mr. Singh’s Pain, Part 2, 2008/2009 ................108


Part 2, 2008/2009 .....................................................................................................109

Figure 11. Total note contributions, Group 1, Mary’s Misery, 2008/2009 .................................110

Figure 12. Build-on note contributions, Group 1, Mary’s Misery, 2008/2009 ............................110

Figure 13. Social network analysis of who built-on whose notes, Group 1, Mary’s Misery,

2008/2009 ................................................................................................................111

Figure 14. Total number of notes read, Group 1, Mary’s Misery, 2008/2009.............................111

Figure 15. Social network analysis of who read whose notes, Group 1, Mary’s Misery,

2008/2009 ................................................................................................................112

Figure 16. Total note contributions, Group 1, Judy’s Last Days, Part 1, 2008/2009 ..................113

Figure 17. Build-on note contributions, Group 1, Judy’s Last Days, Part 1, 2008/2009.............113

Figure 18. Social network analysis of who built-on whose notes, Group 1, Judy’s Last

Days, Part 1, 2008/2009 ..........................................................................................114

Figure 19. Total number of notes read, Group 1, Judy’s Last Days, Part 1, 2008/2009 .............114

Figure 20. Social network analysis of who read whose notes, Group 1, Judy’s Last

Days, Part 1, 2008/2009 ..........................................................................................115




Days, Part 2, 2008/2009 ..........................................................................................117


xv

Figure 25. Social network analysis of who read whose notes, Group 1, Judy’s Last

Days, Part 2, 2008/2009 ..........................................................................................118

Figure 26. Total note contributions, Group 2, Mr. Singh’s Pain, Part 1, 2008/2009 ...................119


2008/2009 ................................................................................................................119

Figure 28. Social network analysis of who built-on whose notes, Group 2, Mr. Singh’s

Pain, Part 1, 2008/2009 ...........................................................................................120

Figure 29. Total number of notes read, Group 2, Mr. Singh’s Pain, Part 1, 2008/2009 ..............120


Part 1, 2008/2009 .....................................................................................................121

Figure 31. Total note contributions, Group 2, Mr. Singh’s Pain, Part 2, 2008/2009 ...................122


2008/2009 ................................................................................................................122

Figure 33. Social network analysis of who built-on whose notes, Group 2, Mr. Singh’s

Pain, Part 2, 2008/2009 ...........................................................................................123

Figure 34. Total number of notes read, Group 2, Mr. Singh’s Pain, Part 2, 2008/2009 ..............123


Part 2, 2008/2009 .....................................................................................................124

Figure 36. Total note contributions, Group 2, Mary’s Misery, 2008/2009 .................................125

Figure 37. Build-on note contributions, Group 2, Mary’s Misery, 2008/2009 ............................125


2008/2009 ................................................................................................................126

Figure 39. Total number of notes read, Group 2, Mary’s Misery, 2008/2009.............................126


2008/2009 ................................................................................................................127




Days, Part 1, 2008/2009 ..........................................................................................129


Figure 45. Social network analysis of who read whose notes, Group 2, Judy’s Last Days,

Part 1, 2008/2009 .....................................................................................................130




Days, Part 2, 2008/2009 ..........................................................................................132


xvi

Figure 50. Social network analysis of who read whose notes, Group 2, Judy’s Last Days,

Part 2, 2008/2009 .....................................................................................................133

Figure 51. Group 1, Mr. Singh’s Pain, Part 1 ..............................................................................134


Figure 53. Group 1, Mary’s Misery .............................................................................................135

Figure 54. Group 1, Judy’s Last Days, Part 1 ..............................................................................135




Figure 58. Group 2, Mary’s Misery .............................................................................................137



Figure 61. Social network position/power map Group 1, Mr. Singh, Part 1, Build-on notes. ....165

Figure 62. Social network position/power map Group 1, Mr. Singh, Part 2, Build-on notes. ....167

Figure 63. Social network position/power map Group 1, Mary’s Misery, Build-on notes. ........169

Figure 64. Social network position/power map Group 1, Judy’s Last Days, Part 1,

Build-ons. ................................................................................................................171


Build-ons. ................................................................................................................173

Figure 66. Social network position/power map Group 2, Mr. Singh’s Pain, Part 1,

Build-ons. ................................................................................................................176

Figure 67. Social network position/power map Group 2, Mr. Singh’s Pain, Part 2,

Build-ons. ................................................................................................................178

Figure 68. Social network position/power map Group 2, Mary’s Misery, Build-on notes. ........180


Build-ons. ................................................................................................................182


Build-ons. ................................................................................................................184

Figure 71. Exemplar 1: 2008/09, Group 1, Mr. Singh’s Pain, Part 1. .........................................200



Figure 74. Exemplar 4: Group 1, Judy’s Last Days, Part 2. ........................................................202





xvii


Figure 80. Exemplar 10: Group 1, Judy’s Last Days, Part 2. ......................................................206






Figure 86. Exemplar 16: Group 1, Judy’s Last Days, Part 2—Reflections .................................209




xviii

DEDICATION

This study is dedicated to the memory of my parents, Irving and Bella Goldstein, and

with never-ending love to my husband, Gary, and my three children,

Ryan, Ilyse, and Isaac Lax.

Knowledge Building in Continuing Medical Education … 1

CHAPTER 1

INTRODUCTION

1.1 Overview

Continuing medical education has been characterized as didactic and ineffective (Abrahamson et

al., 1999; Davis, 2011; Davis et al., 1999; Mazmanian & Davis, 2002; Miller et al., 2008) and in

process of “transitioning from an instructor-centric to a learner-centric model” that includes a

necessary shift from “time-based to value-based” systems (Dorman & Miller, 2011, p. 1339;

Pisacane, 2008). This thesis explores the use of Knowledge Building theory, pedagogy, and

technology, as defined by Scardamalia and Bereiter (2003a, 2006), to test a different model for

physician engagement, that is broader and potentially more expansive—one that places emphasis

on sustained and creative work with ideas—yet is consistent with professional responsibilities of

life-long learning.

Several important conceptual changes in continuing medical education are implied by the

Knowledge Building model—changes that extend the traditional approach through engagement

in (a) collective responsibility for group achievements rather than exclusive focus on individual

advancement and (b) work in design-mode, with ideas treated as objects of creation and

assemblage into larger wholes and new applications, with extension beyond belief-mode where

evidence-based acceptance or rejection of beliefs dominates. The goal is to engage physicians in

“cultures of participation” where individual learning and collective knowledge invention, or

metadesign, advance in parallel (Fischer, 2010, p. 168). This type of knowledge work is aimed at

deliberate and sustained innovation.

The goal of this thesis is to show improvements according to both traditional and non-traditional

measures—to show that there is not tradeoffs but rather improvements on both fronts. This thesis

will describe (a) traditional learning outcomes based on individual measures, (b) performance

over and above traditional measures (i.e., beyond learning as traditionally conceived and

measured), and (c) sociocognitive dynamics that enable work over and above traditional

learning.


Detailed examination of the final year of this study will describe sociocognitive dynamics

through analyses of social network relationships in a Knowledge Building community dedicated

to knowledge improvement. Results demonstrate how a Knowledge Building approach can lead

to more expansive knowledge gains, over and above those predefined by learning objectives in

the curriculum. Making explicit complex relationships embedded in social networks structures

that support a Knowledge Building community is intended to provide more robust understanding

of collective knowledge construction in continuing medical education, as well as corresponding

measures, indicators, and attributes of community engagement.

1.2 Context

This study was conducted in the context of a University of Toronto continuing medical education

course for family physicians, called the End-of-Life Care Distance Education Program. Analyses

were performed across five years of the program, between 2004 and 2009. In-depth comparative

analysis of the social network structures of two groups, consisting of 10 modules, in the last year

of the study (2008/2009), was also performed. In addition, 40% of the 2008/2009 within-note

discourse was analyzed to verify participant structures, determine thematic content, and assess

strength of Knowledge Building indicators. A mixed-methods, (case study methodology was

used to determine if socially mediated processes underlying Knowledge Building improved

physicians’ knowledge, and if so, what social network structure and power relationships

supported knowledge improvement, democratization of participation, and curriculum cocreation

in continuing medical education.

Many family physicians in current practice have not had the opportunity to study palliative care.

Until recently most Canadian and American schools did not include palliative and end-of-life

care in their curricula (Sullivan et al., 2004). Over the past decade or so, this issue has been

addressed in undergraduate, graduate, and continuing medical education.

Numerous papers (Singer & Bowman, 2002; Singer, Martin, & Kelner, 1999; Sullivan et al.,

2004; WHO, 1990), provided necessary background on the state of palliative care education and

led to a series of studies and reports directed toward effecting change and implementation.

Canada’s Romanow Commission (Romanow, 2002) recommended implementation of palliative

care education to support the growth of an aging Canadian population. Since palliative care, had


not been formally taught in most undergraduate medical education curricula, funding was

provided to the provincial governments to address this issue, and the Framework for a National

Strategy on Palliative and End-of-Life Care (Quality End-of-Life Care Coalition of Canada,

2005) emerged.

The End-of-Life Care Distance Education Program was designed to meet this need and fill this

gap in continuing medical education for physicians in the Toronto, York, and Simcoe regions. It

was developed, implemented, and is sustained by funding from the Ontario Ministry of Health

and Long-Term Care, through the Temmy Latner Centre for Palliative Care, Mount Sinai

Hospital, Toronto, Ontario, Canada and offered through the University of Toronto, Faculty of

Medicine, Office of Continuing Education and Professional Development. The Program is

accredited by the College of Family Physicians of Canada Maintenance of Proficiency program

(Mainpro®

) for annual maximum continuing medical education credits (25 Mainpro credits).

Numerous other programs in palliative and end-of-life care have recently been implemented

across Canada and the United States and a variety of research reports from all levels of medical

education have been published about undergraduate medical education (Wear, 2002; Wood,

Meekin, Fins, & Fleischman, 2004), postgraduate residency (Liao, Alpesh, & Rucker, 2004;

Porter-Williamson, von Gunten, Arman, et al., 2004; Weissman & Block, 2002), and physicians’

practice (Bradley et al., 2004; Brennan, 2002). Recommendations on how to improve aspects of

palliative care, such as current pain treatments, patient/family communication, and

interdisciplinary care (Desa et al., 2008; Fineberg, Wenger, & Forrow, 2004; Morrison & Meier,

2004) are also evident in the literature. Many courses are offered in person and some are offered

through web-based learning (Pereira et al., 2008).

It is noteworthy that the online pedagogic design of the End-of-Life Care Distance Education

Program (End-of-Life Care Distance Education, 2004) used in this study was based on

Knowledge Building theory and employs supporting Knowledge Forum®

technology. The

educational design is framed by predefined learning objectives and opportunities for open-ended,

emergent discourse around participant-identified ideas and issues. The five online modules are

structured around palliative care cases; (media clinical scenarios, digital resources, evaluation

components are embedded within. A palliative care expert facilitates collective discourse

amongst the group of family physicians. Asynchronous discussion extends over a one-month


time frame in each of the five online modules. More detailed description of the pedagogic design

of the program is described elsewhere (Lax, Singh, Scardamalia, & Librach, 2006). Participant

discourse and social network interactions in the End-of-Life Care Distance Education Program

are based on the physicians’ discourse captured in Knowledge Forum and analysis using tools

embedded in this environment.

1.3 Theoretical Foundations

The theoretical foundations of this study are based on Knowledge Building as elaborated by

Scardamalia and Bereiter (Scardamalia, 2003a; Scardamalia & Bereiter, 2006); on associated

principles (Scardamalia, 2002) and ideas of continually improvable expertise (Bereiter &

Scardamalia, 1993); and on Bereiter’s (2002c) Education and Mind in the Knowledge Age.

Throughout the thesis the term Knowledge Building is capitalized to denote this specific

approach. There are, of course, many other approaches to knowledge building and when the

phrase is used in a more general sense, it is not capitalized.

Knowledge Building theory (Scardamalia & Bereiter, 2003a) and Knowledge Forum technology

(Scardamalia & Bereiter, 2003b, 2006) support a sociocognitive model of knowledge creation.

Knowledge Building is defined as “the production and continual improvement of ideas of value

to a community, through means that increase the likelihood that what the community

accomplishes will be greater than the sum of individual contributions and part of broader cultural

efforts” (Scardamalia & Bereiter, 2003a, p. 1370). Knowledge Building has been framed in

terms of work in design mode, to draw attention to the importance of processes aimed at

knowledge creation and innovation (Bereiter & Scardamalia, 2003, 2007; Scardamalia &

Bereiter, 2005). It is framed by the notion of continual progressive improvement and the

potentiality of what could be—a world view that implies going beyond what is.

The focus of learning on individual knowledge, which has been characterized as what’s in one’s

head, is reframed in Knowledge Building communities in terms of collaborative participation,

collective knowledge-work with ideas and artifacts, out in the world (Bereiter & Scardamalia,

1996; Scardamalia, 1999; Scardamalia & Bereiter, 1994, 2003a). Scardamalia and Bereiter

(2003a) have explained:

Learning is an internal, unobservable process that results in changes of belief, attitude, or

skill. Knowledge building, by contrast, results in the creation or modification of public


knowledge—knowledge that lives “in the world” and is available to be worked on and

used by other people. Of course creating public knowledge results in personal learning,

but so does practically all human activity. (p. 1371)

Knowledge and ideas are seen as not contained within the mind but as an artifact in public space

to be collectively and collaboratively worked on (Bereiter, 2002c; Bereiter & Scardamalia, 1996;

Popper, 1972). Public space for the identification and problematization of ideas has been called a

“problem space” (Newell, 1980, p. 693); for Knowledge Building this could be called a design

space. Knowledge Forum provides an open space for communities to work with knowledge

artifacts.

Unlike most courseware environments, Knowledge Forum enables the representation of

connections between ideas and the interconnectedness of ideas (Hewitt, 2001; Hewitt &

Scardamalia, 1998; Lax, Scardamalia, Watt-Watson, Hunter, & Bereiter, 2010; Scardamalia,

2003a, 2004a). Most courseware environments use conversational threads to link one idea to the

next; Knowledge Forum employs graphical concept map representationality and functionality to

link multiple ideas and rise-above (Scardamalia, 2002), to create meta (higher-level) views and

perspectives. Additionally, Knowledge Building/Knowledge Forum assessment tools go well

beyond courseware read/post participation statistics, to graphically represent interactivity and

interconnectedness, and describe change over time (Teplovs, 2010; Teplovs & Scardamalia,

2007). This combination of statistical and graphic representations of data, particularly networks

of interactivity, make an important contribution to research assessment and embedded

assessment for students. These representations allow the researcher explicit comparison of

information not previously relatable and augment intuitive design (Akin, 2001). Knowledge

Forum has been specifically designed to provide sociocognitive support for Knowledge Building.

Knowledge Building has been described in terms of 12 interconnected principles (Scardamalia,

2002). Sociocognitive and technological determinants are defined for each and are often used, as

in this study to evaluate indicators of Knowledge Building within the discourse. Some of the

principles integral to this research study are: real ideas, authentic problems; improvable ideas;

idea diversity; rise above; epistemic agency; community knowledge, collective responsibility;

democratizing knowledge; symmetric knowledge advancement; constructive use of authoritative

sources, and Knowledge Building discourse. A complete description of each principle can be


found in Scardamalia’s (2002) paper, “Collective Cognitive Responsibility for the Advancement

of Knowledge.”

Two important features of Knowledge Building discourse are evident in the description of these

principles: emergence and intentionality (Scardamalia & Bereiter, 1991, 2006; Scardamalia,

Bereiter, McLean, Swallow, & Woodruff, 1989). Scardamalia and Bereiter (2005) indicated:

Learning and knowledge creation are both emergent processes, sufficiently similar to

suggest that they are the same process, attaining, different levels of result. To say that

they are emergent is to say the output of the process (an advance in personal

understanding, a new theory, a design innovation, etc.) is not a deducible result of the

inputs. Yet all that educators really have in hand to influence learning and development

are inputs and limited control over environmental conditions. …An educational science

for the Knowledge Age must, we believe, treat ideas as real things and treat minds-

whether individual or collective—as dynamic systems. (p. 36)

Connectionist models of learning and development have been used to explain the concept of

emergence as a dynamic, self-organizing system (Bereiter, 2002c). Thus, it is important to design

for emergence—to provide opportunities and open spaces for Knowledge Building, over and

above learning. Many educational environments, concerned only with transmission of facts and

didactic teaching, do not.

Emergent, design mode work is defined as iterative, creative, and reflective work on personally

meaningful and authentic problems (Bereiter & Scardamalia, 2003). Design mode characterizes

the work of engineers, architects, and scientists. It embodies the notion of collective work with

ideas at the edge of current understanding.

Design mode situates ideas at the centre of education for communal Knowledge Building

throughout formal education and lifelong innovativeness (Bereiter & Scardamalia, 2003a). In

this way, Knowledge Building can be seen as a pervasive approach that transcends in-school

learning, beyond formal education and into continuing education, professional development, and

lifelong improvement. Central to the notion of education of a knowledge-creating process is

better integration of belief mode (the mode of learning facts and internalizing beliefs presented

by others) and design mode (the mode of idea improvement, invention, and theorizing).

In this thesis, Knowledge Building is considered a metadesign theory and Knowledge Forum is a

metadesign environment. Metadesign is defined as “Cultures of participation (that) provide all

citizens with means to become cocreators of new ideas, knowledge, and artifacts in personally


meaningful ways” (Fischer, 2010, p. 168). Knowledge Building, framed as metadesign for

continuing medical education, includes learning, but focuses on the cocreation of ideas,

improvement of knowledge, and participation in the design of the educational agenda.

Knowledge Building communities for continuing medical education have the potential to aim

beyond predefined learning objectives (keeping up to date with the new facts and the latest

information), and instead to focus on emergent issues relevant to the community and aimed at

knowledge creation, innovation, and improvement of real ideas and authentic problems (Bereiter

& Scardamalia, 1993a; Scardamalia, 2002; Scardamalia & Bereiter, 1996, 2005).

This thesis challenges current limitations of continuing medical education, framed as individual

learning, and tests the possibilities of sociocognitive work to extend the boundaries towards

democratization of participation, cocreation of ideas, and metadesign of knowledge work. The

next section provides background on the limitations of continuing medical education as

traditionally conceived.

1.4 Background

Conceptualizations of continuing medical education, locally and abroad, are undergoing

reconsideration (University of Toronto, 2011; Wentz, 2011). Documents highlight opportunities

for increased participation through enhanced individual and community efforts. However this

conceptualization stops short of the potentiality captured in the conceptualization a Knowledge

Building community, that goes beyond increasing participation toward deeper sociocognitive

engagement and a broader, more expansive commitment to collectively engaging in the creation

and design of knowledge work, above and over what is typically a predetermined curriculum in a

continuing medical education course. The potential to engage sociocognitively not only as a

participant but as a designer of one’s own continuing medical education experiences is an

important distinction, and the essence of democratization and metadesign Knowledge Building,

as conceived of in this study.

Lifelong learning is the cornerstone of continuing medical education (Abrahamson et al., 1999;

Appelbaum, 2002; Collins, 2009; Manning & DeBakey, 2011). However, recent reports—by the

Macy Foundation (Hager, Russell, & Fletcher, 2007); the Agency for Healthcare Research and

Quality (Marinopoulos et al., 2007); the Conjoint Committee on Continuing Medical Education


(Miller et al., 2008); and more recently and locally, the University of Toronto’s Faculty of

Medicine, Office of Continuing Education and Professional Development’s strategic plan 2011–

2016 (University of Toronto, 2011)—concur: continuing medical education is in need of

substantive change. The Conjoint Committee on Continuing Medical Education (Miller et al.,

2008) has stated:

To provide the best care to patients, a physician must commit to lifelong learning, but

continuing education and evaluation systems in the United States typically require little

more than record of attendance for professional association memberships, hospital staff

privileges, or reregistration of a medical license. (p. 95)

These reports have criticized continuing medical education by pointing to a disconnect between

requirements of licensing bodies and real-world practice. They recommend that continuing

medical education evolve, from counting hours of participation to recognizing physician

achievements in knowledge, competence, and performance. The Conjoint Committee on

Continuing Medical Education recommended that medical boards should require valid and

reliable assessment of physicians’ learning needs, and in doing so, the boards should collaborate

with physician and continuing medical education communities to assure that continuing medical

education achieves maximal benefit for physicians and patients—to assure the discovery and use

of best practices for continual professional development and maintenance of competence (Miller

et al., 2008). Many reports recommend that research in continuing medical education should be

raised to a national priority.

The current study addresses the challenge of this new agenda by pushing it even further, focusing

not only on work with best-practice knowledge, but on intentional work to move beyond best-

practice knowledge (Scardamalia, 2002). Reframing continuing medical education within a

Knowledge Building approach requires investment over and above learning—an intentionally

directed toward at sustained improvement of ideas, expertise, and patient care— that has the

potential to elevate community knowledge as well as individual achievements.

Reframing continuing medical education as continually improvable Knowledge Building

(Scardamalia & Bereiter, 2003a; Bereiter, 2002), may also require changes in attitude and

perceptions (Guest, Regehr, & Tiberius, 2001; Mylopoulos & Scardamalia, 2008). Moving from

a highly individualized culture primarily based on self-evaluation (Dunikowski, 2011; Campbell

& Parboosingh, 2011) towards a culture of community participation in sociocognitive knowledge


work, and metadesign of curriculum (Fischer, 2009a, 2009b, 2010; Fischer & Giaccardi, 2006;

Fischer & Konomi, 2007; OECD, 2007), may require a deeper and broader commitment to

maintenance of competence (Brennan, 2002; Collins, 2009).

Physician competence is currently defined by the Royal College of Physicians and Surgeons of

Canada in the CanMEDS roles framework (Frank, 2005, p. 3), which puts the medical expert at

the centre, surrounded by the roles of scholar, health advocate, manager, collaborator,

communicator, and professional. The role of scholar emphasizes “four critical concepts: lifelong

learning/CPD, critical appraisal, research literacy, and teaching others” (Frank, 2005, p. 8). It

makes explicit the inclusion of teaching and research and the “ethical obligations for lifelong

maintenance of competence” (Frank, 2005, p. 8).

Nowhere in the CanMEDS framework is there mention of responsibility toward improvement of

knowledge artifacts of value to the community, and responsibility to work toward improvement

of the profession, as defined by pervasive Knowledge Building (Scardamalia, 2002; Scardamalia

& Bereiter, 2003a). Continuing medical education is thus conceived of as a one-way enterprise,

from transfer from those who know to those who don’t; as learning. There seems to be little or

no commitment or responsibility to knowledge improvement, beyond personal acquisition. The

potential for Knowledge Building is yet unrealized in any broad way in continuing medical

education. Knowledge Building democratization of continuing medical education, involving

opportunities for metadesign, cocreation, and collective innovation may require coevolution of

physician competencies, as learners and design agents in the advancement of their profession.

The present study will deal with the potential of Knowledge Building theory and Knowledge

Forum technology to support collective knowledge improvement in continuing medical

education, over and above (and yet including) individual learning. Three important contributions

a Knowledge Building theory and technology can make are: (a) support of sustained Knowledge

Building discourse, which involves cocreation of ideas, linkages between ideas, and movement

of discourse toward higher-level organization of ideas; (b) democratization of ideas, as

demonstrated by centrality of all participant ideas and opportunities for emergent knowledge

work as identified by the community; and (c) new measures of participation based on Knowledge

Building activity, interactivity, and social network relationships, which identify outcomes over

and above individual test scores, and help communities judge their own knowledge


improvements, set goals, and design continuously improvable Knowledge Building

opportunities. The goal of this study is to examine the potential of Knowledge Building in

continuing medical education.

1.5 Purpose

The purpose of this study is to show improvements according to traditional as well as

nontraditional measures—to show that there can be improvements, not tradeoffs, on both fronts.

This thesis will describe (a) traditional learning outcomes based on individual measures; (b)

performance over and above traditional measures (i.e., beyond learning as traditionally

conceived and measured); and (c) sociocognitive dynamics that enable work over and above

traditional learning.

Currently, the structure and paradigm of continuing medical education have sustained some

criticism resulting in a series of recommendations for change that support higher levels of

individual participation and social engagement. This study builds-on recommendations for

greater opportunities for collaborative learning and more online initiatives in continuing medical

education.

Knowledge Building democratization of participation, design, and innovation is proposed. It is

important for physicians practising palliative care to better understand total pain management—

the scientific facts, the medical issues, and the mathematics of titration. However, it is also

important to provide family physicians with different levels of knowledge and experience in

palliative care, with opportunities to identify knowledge lacks, voice their concerns, professional

practice questions, patient problems, and ideas for potential improvement. By providing an open

problem-space, it is posited that participants in the course will become metadesigners, cocreators

of the content, the ideas, the knowledge, and the discourse. Thus, it is anticipated that each year

and in each group the curriculum (the ideas at the centre of the discourse space) will be

somewhat different, depending on the needs, interests, and directions that the participants want to

go in—as identified by the Knowledge Building community as important to them.

This thesis will describe characteristics and relationships in a social network deemed to be

successful by traditional measures, through pre-/posttests, through Knowledge Building

activity/interactivity, and through social network structural and content analyses.


Social network analysis enables descriptions of structural relationships, based on network

density, position/power, centrality, clique, and cohesion measures, as well as sociocognitive

dynamic patterns of discourse, according to relationships between students, students/facilitator,

and knowledge work with facts and/or emergent ideas. A comprehensive Knowledge Building

study involving traditional measures of success and social network structural and content

analyses has not been conducted in continuing medical education.

Evidence of success of educational innovations is often provided through proof of knowledge

gains and positive feedback on program satisfaction surveys. These are important learning

outcomes but are rooted in an individual metaphor. Despite the importance of these measures,

they provide insufficient descriptions to help us understand “how” and “why” innovations

succeeded (Kanter, 2008), and offer no window into necessary representations of socially

mediated interactivity. What is required are elaborated, detailed descriptions of the nuances,

relationships, and interrelationships and the sociocognitive dynamics of Knowledge Building—

to provide better understanding of community engagement and structures that scaffold collective

Knowledge Building.

1.6 Significance

This study is significant because it will explore and clarify Knowledge Building and the

relationship between traditional measures of educational success and social network structural

and content assessments, and how they contribute to the advancement of current

conceptualizations of continuing medical education. Continuing medical education has often

been characterized as an individual endeavour, conceived within the constraints and limitations

of self-assessment, self-directed learning, and self-reflection and didacticism (Davis & Davis,

2010; Davis et al., 2006; Davis et al., 1999; Silver, Campbell, Marlow, & Sargeant, 2008), as

opposed to socially mediated assessment. Many continuing medical education courses are large-

group lectures and/or conference presentations (Davis et al., 1999; Davis, Davis, & Bloch, 2008).

Numerous web-based courses are available for individual online learning (Cook et al., 2008;

Curran, Lockyer, Sargeant, & Fleet, 2006; Fordis et al., 2006). Few provide opportunities for

collective discourse; and even fewer, apart from what I describe in this study, have been

designed specifically to provide opportunities for collective Knowledge Building in continuing

medical education.


This will be the first known study to describe social network structural dimensions and

sociocognitive dynamics of sustained, collective Knowledge Building in continuing medical

education; it will describe a social network of physicians working with knowledge and ideas in

an online problem space, over and above accomplishing predetermined learning objectives; and,

as well, corresponding assessments of traditional outcome measures and social network

structural and content analyses of Knowledge Building. This study aims to describe socially

mediated relationships and provide evidence of the potential of Knowledge Building in

continuing medical education.

1.7 Research Questions

This study will be guided by the following research question and a set of subquestions. The

research question is:

Does Knowledge Building improve physicians’ knowledge and understanding of

palliative care in a Web-based continuing medical education course, and if so, what

social network structural relationships and sociocognitive dynamics support knowledge

improvement, and contribute to democratization and metadesign of Knowledge Building

in continuing medical education?

Subquestions organized in the following three clusters will frame this study:

Cluster 1 Traditional outcome measures

Cluster 2 Performance over and above traditional measures (beyond learning as

traditionally conceived and measured)

Cluster 3 Sociocognitive dynamics that enable work over and above traditional

learning

1.7.1 Research Clusters and Subquestions

Cluster 1: Traditional outcome measures.

1. Did students’ pain knowledge improve from pre to posttest and if so, is this

improvement significant and what is the effect size?


2. Traditionally continuing medical education is a self-directed or didactic experience;

what are participants’ attitudes and opinions toward collaborative online knowledge

building in the End-of-Life Care Distance Education Program?

Cluster 2: Performance over and above traditional measures (i.e., beyond learning as

traditionally conceived and measured).

3. What are participants’ online activity and interactivity measures, by module, by

group, and by year, in the 2004–2005 pilot and 2005–2009 study, using the

Knowledge Forum analytic toolkit (ATK)?

4. What participation patterns emerge in the 2008/2009 Group 1 and Group 2 (selected

for analysis because of largest pre-/posttest knowledge gains), through use of the

graphical ATK contribution build-on/read assessment tool and corresponding social

network analyses of network density? Is there a significant difference in 2008/2009

Group 1 and Group 2 pain knowledge scores from pre- to posttest and is there a

significant difference between groups? If so, what is the relationship between these

differences and patterns of social network density?

5. What are the social network structural differences between groups that support

increased knowledge improvement and how are these differences related to

centralization, cohesion, and/or democratization of participation and ideas? Is there a

relationship between social network dimensions?

Cluster 3: Questions regarding the sociocognitive dynamics that enable work over and above

traditional learning.

6. What are the social network relationships between structural position, power—

defined as centrality of ideas—and knowledge improvement; how are these

relationships reflected in facilitator/student sociocognitive dynamics, through

emergent themes within the discourse (beyond learning objectives), in the complexity

of discourse, and by indicators of Knowledge Building?

6.1 What is the relationship between facilitator and students position/power? Whose

ideas are at the core, middle, and periphery of the space?

6.2 Can the facilitator’s stance be described as a monitor, mentor, partner or expert

in the sociocognitive knowledge work? Knowledge work is advanced through


statements and questions. Who asks the questions that drive knowledge work—

students or the facilitator?

6.3 Is there evidence of metadesign knowledge work, as expressed through

emergent themes/ideas, beyond the predefined learning objectives? What

themes emerge?

6.4 Is discourse clinically complex as defined by elaboration or compilation of

knowledge and ideas?

6.5 Is there evidence, in the discourse, of Knowledge Building indicators such as

community responsibility for knowledge advancement, democratization of

knowledge, epistemic agency, improvable ideas, and work with real ideas,

authentic problems? And are indicators of Knowledge Building metadesign

present, as evidenced by intentionality and strength of epistemic agency, work

with community knowledge, and emergent ideas?

The goal of this research study is to demonstrate improvements according both traditional

learning and nontraditional Knowledge Building measures and to provide a meaningful

description of Knowledge Building, in a socially mediated discourse space, and the potential for

change in traditional power structures, in favour of democratization of ideas, cocreation of

knowledge, and metadesign of continuing medical education.

1.8 Dissertation Organization

This study follows traditional research reporting structures of introduction, literature review,

methods, results, discussion, and conclusions. Chapter 1 sets out the background, purpose, and

significance of the study and provides the overarching structure, based on the three clusters of

research subquestions, as delineated above. Chapter 2 is a review of the literature and is also

divided into three sections corresponding to the study clusters. The first section provides an

overview of current issues in continuing medical education, the second reviews key concepts of

Knowledge Building and studies employing social network analyses, and the third section

summarizes the literature on social network analyses relevant to the context herein. Chapter 3

describes the study methods; it begins with an overview of the case-based, (media pedagogic

design of the End-of-Life Care Distance Education program; this section is followed by details of

the research methods. Research methods are divided into three sections, based on the conceptual


categories set out in the clusters of research subquestions. Chapter 4 presents results of the pilot

study. Chapters 5, 6, and 7 provide results of the research subquestions based on the three

clusters. Thesis results are discussed in Chapter 8, and conclusions in Chapter 9 (see Table 1 for

a summary of all the above). Validation of pre-/posttest questionnaires can be found in the

Appendices.


Table 1

Research Clusters and Organization of Dissertation

Introduction and

Research

Clusters

Literature

Review

Methods

Results

Discussion

and

Conclusions

CLUSTER CHAPTER 1 CHAPTER 2 CHAPTER 3 CHAPTER 4 (pilot study) CHAPTERS 8/9

CHAPTER 5

1 Traditional

measures

Continuing

medical

education

Pre-/posttests, attitudes and opinions survey

across all years, 2005–2009

Results of knowledge gains, attitudes, and

opinions, 2005–2009

Synthesis and

significance

CHAPTER 6

2 Performance

over and above

traditional

measures

Knowledge

Building

and social

network

studies

Activity, interactivity measures across 5

years and in-depth SNA using KF/ATK

measures of two groups in final year,

2008/2009; SNA of centrality, cliques,

cohesion using Netminer 3 software

Results of read/write activity, build-on

interactivity, across years and comparative

results of social network density of notes

built-on and read in final year, 2008/2009;

comparative results of centrality measures

with and without the facilitator across all

2008/2009 modules

Synthesis and

significance

CHAPTER 7

3 Sociocognitive

dynamics that

enable work

over and above

traditional

learning

Knowledge

Building

and social

network

analysis

(SNA)

SNA of position centrality/power of ideas

and within-note analyses

⋅ Facilitator stance and examination of

who asks questions that drive the

discourse? (Demonstrates intentional

KB/epistemic agency and collective

responsibility)

⋅ Themes identification beyond learning

objectives

⋅ Complexity of clinical discourse

⋅ Modules to be scored for evidence of

Knowledge Building indicators

Results of position centrality/power of

ideas in relationship to knowledge gains

- Facilitator/student structures, facilitator

stance as partner and expert

- Emergent ideas and themes are identified

beyond learning objectives

- Discourse is categorized as elaborated

and compiled.

- Principles/indicators of Knowledge

Building are scored/evident; exemplars are

provided

Synthesis and

significance


CHAPTER 2

LITERATURE REVIEW

2.1 Introduction

Recent research and reports on the ineffectiveness of continuing medical education and calls for

reform, new ideas, new applications, and alternative strategies for improvement were described

in the first chapter and will be elaborated upon in this chapter, the literature review.

Continuing medical education courses, for physicians in practice, are typically designed for the

transmission of facts, new treatments, recent discoveries, and understanding of best-practice

guidelines, aimed at improving physician practice and patient care. There is no doubt that the

transmission of the most-up-to date knowledge is essential. However, this thesis argues that the

knowledge transmission approach is limiting and suggests that in addition to learning the latest

facts and knowledge of best practice guidelines, physicians should be provided with

opportunities to go beyond best practices (Scardamalia, 2002), through sociocognitive

Knowledge Building.

The goal of this thesis is to show improvements according to both traditional and nontraditional

measures. As noted previously, this thesis will describe (a) traditional learning outcomes based

on individual measures, (b) performance over and above traditional measures (i.e., beyond

learning as traditionally conceived and measured), and (c) sociocognitive dynamics that enable

work over and above traditional learning. The literature review has been structured around these

three research clusters, to inform an understanding of each area and identify knowledge and

research gaps. The first section will provide necessary definitions and background on continuing

medical education. The second section will provide an overview of Knowledge Building theory,

and the third section will review social network analysis and other strategies of network

analyses. The literature review will conclude with a summary of implications for Knowledge

Building in continuing medical education.


2.2 Cluster 1: Traditional Conceptualizations of Continuing Medical Education

A comprehensive history of the evolution of continuing medical education in Canada has

recently been published by Dr. W. Dale Dauphinee (2011) tracing the origins of concepts and the

evolution of structures. Chapters from the same book (Wentz, 2011) provide details too

numerous for inclusion here, on such subjects as accreditation of continuing medical education

(Woollard, 2011), standards for specialists defined by the Royal College of Physicians and

Surgeons of Canada (Campbell & Parboosingh, 2011); and standards, and for family physicians

as defined by the College of Family Physicians of Canada (Dunikowski, 2011).

The first section of the literature review will provide an overview of traditional

conceptualizations of continuing medical education and current issues, important to this research

study.

2.2.1 Continuing Medical Education for Individual Lifelong Learning

2.2.1.1 Definition. The Canadian Institutes of Health Research (2003) have defined continuing

medical education as “any and all ways by which physicians learn and maintain their

competence” and specifically refers to “education after certification and licensure” (Davis,

Evans, et al., 2003). This definition of continuing medical education is often extended, drawing

relationships to professionalism and practice improvement, through two other commonly used

terms, continuing professional development and knowledge translation (Davis, Barnes, & Fox,

2003; Davis, Evans, et al., 2003; Graham et al., 2006; Wentz, 2011). Continuing professional

development “embodies both professional learning and personal growth” (Davis, Evans, et al.,

2003). Some definitions distinguish continuing medical education from professional

development by including concepts of performance improvement through learning in clinical

practice settings (Davis, Barnes, et al., 2003; Bennett et al., 2000). Often, notions of continuing

medical education and continuing professional development are linked together, as in the

recently revised American Medical Association (2010) definition, which is as follows:

CME consists of educational activities which serve to maintain, develop, or increase the

knowledge, skills, and professional performance and relationships that a physician uses to

provide services for patients, the public or the profession. The content of CME is the

body of knowledge and skills generally recognized and accepted by the profession as

within the basic medical sciences, the discipline of clinical medicine and the provision of

health care to the public. (p. 2)


Evaluation of knowledge translation is beyond the scope of this study. Knowledge translation is

defined and distinguished as “the exchange, synthesis and ethically sound application of

knowledge . . . to accelerate the capture of benefits of research . . . through improved health,

more effectives services and products and a strengthened health care system” (Davis, Evans, et

al., 2003). The definition of knowledge translation links continuing medical education to health

care outcomes (Davis, 2011; Dorman & Miller, 2011).

The College of Family Physicians of Canada (n.d.) has stated: “As professionals, family

physicians are obliged to remain current on advances and trends in medicine and health care

delivery. This is achieved through participation in a variety of activities that constitute

continuing professional development (CPD)” (para. 1). The End-of-Life Care Distance

Education Program received full accreditation from The College of Family Physicians of Canada

and was further reviewed and accepted by University of Toronto’s Office of CEPD at the Faculty

of Medicine, which uses the following definition: “Continuing education is a process of lifelong

learning for health professionals to expand and fulfill their potential, retain their capacity to

practice safely and effectively, and to meet the healthcare priorities of the population and broader

health systems” (University of Toronto, 2011, p. 6).

The Office of CEPD recently published a strategic plan for 2011–2016 called “Better Education,

Better Outcomes” (University of Toronto, 2011). The plan calls for a change that includes

moving from self-learning and didactic formats to collaborative learning and social network

opportunities, so as to better integrate and sustain continuing medical education for lifelong

learning. This shift is intended to move from episodic learning, i.e. discrete educational events

such as lectures and workshops at conferences, to longitudinal, continuous interactions, such as

those involving online social network communities and practice-based settings.

Strategic directions identified by the 2011–2016 plan are:

1. Enhance best practice and faculty development for continuing education

2. Advance research, innovation, and scholarship

3. Strengthen practice-based education

4. Foster quality improvement in continuing education

5. Promote patient and public engagement (University of Toronto, 2011, pp. 7–9)

Enabling strategies include entrepreneurship, information technology and social media,

communities of practice, partnerships, and sustainable funding (University of Toronto, 2011, pp.


10–11). It is clear that technology and concepts of communities to advance knowledge figures

prominently in this new vision.

Approximately 27,000 individuals participated in various events/programs in 2009–2010 offered

by the Office of CEPD; however, as the aggregate data over five years demonstrates, most

participated in live events and very few participated in web-based learning (University of

Toronto, 2011, p. 4, fig. 2). Of the online courses offered most are designed for individual

learning; few are designed for online collaborative discourse.

The new vision embodied by the Faculty of Medicine’s strategic plan is well aligned with the

concept of knowledge-creation concepts of Knowledge Building: “The Office of CEPD is

seeking to change problem solving into possibilities for breakthrough thinking. This included

thinking about continuing education as a type of creative experiment” (University of Toronto,

2011, p. 1).

2.2.1.2 From individual to socially mediated participatory metadesign. Over the last decade

or more, e-learning continuing medical education courses have become popular. Web portals

such http://www.MDcme.ca offer numerous online courses in various clinical content areas. A

review of this site indicates that most courses are individually accessed online and aimed at

learning (Knowles, 1975); few use discussion board environments or are aimed at computer

supported collaborative learning (Brown & Duguid, 2000); none were found that focus on

collaborative Knowledge Building (Scardamalia & Bereiter, 2003a).

Various e-learning and information technology tools for individual physician use have been

developed by Canadian regulatory bodies. In 2000, the Royal College of Physicians and

Surgeons of Canada upgraded their Maintenance of Competence (MOCOMP) program and

established a mandatory Maintenance of Certification (MOC) program. Physicians were required

to document their continuing professional development activities and to facilitate this, the

College developed a Web Diary and a Question Library (Campbell & Parboosingh, 2011). A few

years later these two programs were amalgamated into one site called Mainport for individual

physicians’ record keeping and MOC credits.

Similarly, the College of Family Physicians of Canada has sponsored various innovative

programs using information technology, such as the eCME Resource Centre (Dunikowski,


2011). This resource site makes available dozens of online course. Two well-established web-

based programs sponsored by the College of Family Physicians are the Self Learning™ suite and

the Pearls™ (n.d.) program. Self Learning™ is web published in issues, each of which contains a

series of multiple-choice questions and short-answer management problems; a concise summary

highlights major points, with links to PubMed abstracts and full text articles. Self Learning™

had more than 6,000 subscribers worldwide in 2009 (Dunikowski, 2011). Pearls was also

introduced in a web-based version in 2009. It is “a self-directed, structured, learning activity”

(Dunikowski, 2011, p. 246) that can be completed by physicians as an evidence-based practice

reflection exercise:

Participants use the medical literature to answer their own practice questions and later

reflect on how well decisions have been integrated into their practice. Based on Donald

Schon’s reflective learning cycle, a Pearl’s exercise has five steps: formulating a practice-

based question, seeking information from peer-reviewed literature, evaluating and

critically appraising the articles, making a practice decision based on what was learned,

and indicating the effect of this decision on patient outcome. (Dunikowski, 2011, p. 246)

Physicians are awarded Mainpro-C credits for Pearls exercises. Concerns about self-directed

learning and self-evaluation in these types of continuing medical education scenarios have been

published (Regehr & Eva, 2006) and prompted the notion of “directed self-assessment” by the

Committee on Accreditation of Continuing Medical Education (ACCME, n.d.; Woollard, 2011),

and more recent recommendations for collective knowledge work and community engagement,

as in the Office of CEPD strategic plan (University of Toronto, 2011), discussed above.

Ill-structured knowledge domains are conceptually complex and irregular across cases (Spiro,

Feltovich, Jacobson, & Coulson, 1991). It is well known that medicine has been characterized as

an ill-structured domain (Spiro, Coulson, Feltovich & Anderson, 1998; Spiro et al., 1991).

Interactions made possible by computers has been identified as the ideal medium for criss-

crossing ill-structured domains (Spiro & Jehng, 1991). Although Spiro and Jehng (1991) were

not referring to Knowledge Building in Knowledge Forum, the notion of benefits from

interactivity and linking of ideas in an ill-structured domain is clear. Connecting ideas is most

productive when socially constructed and aimed at improvement of ideas (Scardamalia &

Bereiter, 2003a, 2003b).

As we have seen from the literature reviewed above, physicians usually do not have the

opportunity to collectively participate in Knowledge Building—to create and design their own


collective continuing education experiences. Physicians’ contributions to the metadesign of their

continuing medical education experiences may allow these experiences to become more relevant

and emergent, to better meet their needs, scaffold continuous improvability, and elevate

community knowledge. Various studies in other contexts have indicated that metadesign

translates into improved practice and care—addressing the knowledge to practice gap (Fischer,

2009a, 2009b, 2010; Fischer & Giaccardi, 2006; Fischer & Konomi, 2007).

The notion of improving physician participation in continuing education is not new; what is new

is the idea of elevating physician participation in continuing education as a metadesigner of the

curriculum and cocreator of knowledge, with the intended purpose of improving ideas of value to

the community. This may be possible in an environment that supports collective knowledge

building, where continuing medical education can assume an open problem space, a more

emergent structure, and more symmetrical opportunities to engage in sociocognitive work in

belief and design modes, as elaborated below. It is posited that through theory-based designs

within a knowledge building perspective, continuing medical education may become more

authentic and democratized, promoting changes in performance and idea innovation.

In the next section I will review selected literature on Scardamalia and Bereiter’s theory and

concepts of Knowledge Building. It is important to note that there is a much broader literature on

knowledge building that is not reviewed, as it is beyond the scope of this paper. To put this

broader literature in perspective, a web search of the phrase “knowledge building” a decade ago

showed less than 10,000 entries. Within five to years it had reached the same level as

“knowledge creation”—its more popular synonym in the business world: about 500,000 entries.

It has continued to grow as an area of intense research, keeping pace with its counterpart,

knowledge creation, and now shows well over a million links.

There seems to be growing awareness that knowledge creation must extend beyond the realm of

intellectual elites—that citizens worldwide need to have a part in a knowledge-creating culture.

Giving them a role requires something that expands the idea of knowledge creation and the

notion of individual genius often associated with it. That challenge is being tackled by a

worldwide community, as suggested at the Computer-Supported Collaborative Learning

Conference held in 2011 in Hong Kong (CSCL2011, 2011). Knowledge building was one of five

major strands, with over 50 research reports presented. Knowledge Building was earlier


represented in the Cambridge Handbook of the Learning Sciences (Sawyer, 2006) as one of five

foundational approaches within the learning sciences. The strong collaborative framework for

this work is reflected in special issues of the Canadian Journal of Learning and Technology

(Scardamalia & Bereiter, 2010) devoted to knowledge building and QWERTY: Interdisciplinary

Journal of Technology and Culture, with approximately 12 disciplines represented, as well as

multiple cultural contexts and research involving students from elementary to tertiary education,

teacher education, and teacher-researcher-government partnerships. Topics range over

knowledge building theory, models, and theory-based principles; tools and methodologies;

knowledge-building dialogue; applications (e.g., Aboriginal education and totally online and

blended learning); and results and outcomes (including a quantitative model for the analysis and

the assessment of the collaborative structure of a knowledge-building community and assessment

of community knowledge). There is an Italian Association for Knowledge Building, a Centre for

Research on Knowledge Building and Technology at the University of Helsinki, and major

knowledge-building research programs spanning the Americas, Europe, and Asia.

2.3 Cluster 2: Performance Over and Above Traditional Individual

Outcomes, Through Collective Knowledge Building

This research cluster has its basis in the literature of collective Knowledge Building, as defined

by Scardamalia and Bereiter (2002, 2003a, 2006). Knowledge Building theory, pedagogy, and

technology demonstrate the theme of this second cluster: performance over and above traditional

outcomes (beyond learning as traditionally conceived and measured), through collective

Knowledge Building. This section begins with a review of selected Knowledge Building studies

in medical and health sciences education that demonstrate the challenges of research in this

context.

2.3.1 Knowledge Building Research in Medical and Health Sciences Education

Various Knowledge Building research studies have been conducted in medical and health

sciences education; however, a paucity of research exists particularly in the area of continuing

medical education.

Two recent doctoral research studies conducted by Punja (2007) and Mylopolous (2007)

examined Knowledge Building in the context of medical education. Punja’s study involved four


case studies across three different levels of medical education: undergraduate medical education,

graduate residency, and postlicensure continuing medical education. Punja conducted a brief

pilot study in a psychiatric continuing medical education course. Knowledge Forum was used as

an add-on to support additional discussion, beyond face-to-face sessions; the discourse never

built momentum. Punja found similar results across all settings, characterizing each as teacher

dominated, and concluding that teacher-generated activities made it difficult for students to

assume epistemic agency for their knowledge work. Barriers to Knowledge Building included

the belief that knowledge innovation should occur after acquisition of foundational knowledge,

as opposed to operating in parallel and providing a better basis for acquisition of foundational

knowledge.

Similar barriers were identified in Mylopoulos’s (2007) grounded theory dissertation study and a

related publication (Mylopolous & Scardamalia, 2008). Mylopolous conducted semistructured

interviews with a sample of 15 clinical faculty members at University of Toronto’s Faculty of

Medicine. She demonstrated that clinical faculty held the view that innovation is developed

through research and then diffused through the community for adoption. Hence, in this view,

innovation is removed from personal responsibility and daily practice. Mylopolous and

Scardamalia concluded that these perceptions of innovation typically limited participants’

engagement in collaborative processes central to knowledge building.

Studies such as these inform our understanding of how to better design the social infrastructure

critical to support collective community online work (Bielaczyc, 2006). Bielaczyc (2006)

described four dimensions: “(a) cultural beliefs, (b) practices, (c) socio-techno-spatial relations,

and (d) interaction with the “outside world” (p. 301). Both Punja’s (2007) and Mylopoulos’s

(2007) studies point to the substantial challenges that need to be addressed to change perceptions

and engagement with knowledge innovation.

Some other recent studies in the health sciences explored the potential of Knowledge Building:

in a pharmacy course, typically taught through lectures (Sibbald, 2009): in a hospital-based

interprofessional health sciences teams (Russell, 2002, 2005); and in an undergraduate

interprofessional health sciences education pain curriculum (Lax, Scardamalia, Watt-Watson,

Hunter, & Bereiter, 2010; Lax, Watt-Watson, Pennefather, Hunter, & Scardamalia, 2002, 2003a,

2003b).


Each of these studies uncovered substantial challenges and the need for research demonstrating

that there need not be tradeoffs between learning and knowledge building—but rather these

forces may be mutually reinforcing. And none of these studies examined social network

relationships and sociocognitive dimensions of sustained knowledge building in continuing

medical education. The current study addresses this gap in the literature, to illuminate the

community relationships that support knowledge improvement beyond individual knowledge

growth. What follows is a detailed overview of the theory, pedagogy, and technology of

Knowledge Building.

2.3.2 Distinction Between Learning and Knowledge Building

Central to education becoming a knowledge-creating process is better integration of belief mode

(argumentation, persuasion, and critical analysis) and design mode (theorizing, invention, and

idea improvement; Bereiter & Scardamalia, 2003b). Currently, work in design mode is seriously

underrepresented in continuing medical education. Work in design mode is fundamental to

knowledge building. Designers do not contemplate the possibility of a final state of perfection

(Petroski, 2003); neither do knowledge building students and teachers. As Scardamalia and

Bereiter (2003a) have noted, knowledge building is associated with innovation, intellectual

capital, and intellectual property, and is synonymous with knowledge creation. Its growth

alongside knowledge creation reflects a need for understanding processes by which all citizens

can engage in knowledge creation and educational institutions operate as knowledge-creating

organizations.

It is posited that the didactic paradigm of most current continuing medical education needs to

allow for more authentic, relevant, design-mode work. Continuing medical education reframed

as a trajectory of life-long knowledge building may add new understanding to help effect change

and inform us about the sociocognitive dimensions and supports required for the collective

advancement of knowledge. Knowledge Building offers a framework for pedagogic design and

evaluation (Scardamalia & Bereiter, 2006). Knowledge Forum is an Internet-based communally

accessible learning technology developed by Scardamalia and Bereiter to support collaborative

Knowledge Building. Knowledge Building provides a theoretical and pedagogical framework

and Knowledge Forum provides an easily accessible collaborative communication technology to

support educational design and evaluation. This combination of theory, pedagogy, and


technology for design and evaluation allows us to address continuing medical education in

dynamic new ways.

Scardamalia and Bereiter (2003a) have defined Knowledge Building as follows:

Knowledge Building may be defined as the production and continual improvement of

ideas of value to a community, through means that increase the likelihood that what the

community accomplishes will be greater than the sum of the individual contributions and

part of broader cultural efforts. (p. 1370)

Thus, Knowledge Building is seen as pervasive, not limited to education, but applicable to

society on the whole and practice-based communities, such as those addressed in the study

herein.

Scardamalia and Bereiter (2003a) have clearly distinguished learning (as private knowledge)

from Knowledge Building (as public knowledge):

Learning is an internal, unobservable process that results in changes of belief, attitude or

skill. Knowledge building, by contrast, results in the creation or modification of public

knowledge—knowledge that lives ‘in the world’ and is available to be worked on and

used by other people. Of course creating public knowledge results in personal learning,

but so does practically all human activity. (p. 1371)

Knowledge Building is further distinguished from learning by disclaiming the “mind as

container” metaphor and “jug to mug” analogies for teaching and learning (Bereiter &

Scardamalia, 1996, p. 486). In Bereiter and Scardamalia’s (1996) paper, “Rethinking Learning,”

the authors used “the term ‘knowledge building’ to refer to the production of knowledge

objects—objects that in Popper’s scheme occupy World 3” (p. 494). They indicated that

Knowledge Building is activity directed toward changes in World 3 (through ideas, theories, and

interpretation), while learning is directed toward World 2 (states of knowing). With this comes

the notion of “moving ideas to the centre”—a pivotal and vivid conception describing how to

work with knowledge objects and artifacts (Scardamalia, 1999). Using these distinctions,

Bereiter and Scardamalia (1996) have indicated that learning can be seen as complementary to

Knowledge Building. In their article, “Beyond Bloom’s Taxonomy,” Bereiter and Scardamalia

(1999) set out a series of levels of objectification for knowledge work that go beyond Bloom’s

(Bloom, Engelhart, Furst, Hill, & Krathwohl, 1956) original or revised (Anderson & Krathwohl,

2001) educational objectives.


A complete history of Scardamalia and Bereiter’s ideas has been published (Scardamalia &

Bereiter, 2010). Their definition and development of Knowledge Building theory resonate from

years of work in the domains of cognitive psychology and education, guided by philosophical

underpinnings, influenced by new technologies, the Internet, and concerned with rigorous

research and design methods to support innovation in the knowledge age. They and an ever-

expanding international knowledge-building community of teachers, scholars, and researchers

have advanced this work substantially.

2.3.3 Historical and Cultural Relevance of Knowledge Building

In 1989, Scardamalia and Bereiter and their colleagues McLean, Swallow, and Woodruff at the

Centre for Applied Cognitive Science, at the Ontario Institute for Studies in Education (now part

of the University of Toronto), published a seminal paper, “Computer-Supported Intentional

Learning Environments” (CSILE). The authors described principles based on cognitive science

research for the design of computer environments. These computer environments were intended

to scaffold students’ intentional control over their own learning; that is, the setting of cognitive

goals, self-monitoring, strategies for comprehension and organization of knowledge.

Intentional learning and CSILE were described in parallel. CSILE was initially developed as a

hypermedia system using text and graphic notes for groups of students to share their thoughts.

Scardamalia et al. (1989) called this process “building a collective database (knowledge-base)

. . . CSILE stores the thoughts entered by each student, making them available to everyone” (p.

52). Results of early trials using CSILE with university master’s level students provided

encouraging feedback on processing and reprocessing of thoughts on research literature. CSILE

was implemented as a networked computer system, in numerous classrooms at all grade levels.

Scardamalia et al. (1989) provided a review of cognitive research of the day and noted, along

with many researchers, the need for active rather than passive learning strategies. But while

many researchers were calling for intelligent tutoring systems, Scardamalia et al. declared:

It is not the computer that should be doing the diagnosing, the goal-setting, and the

planning, it is the student. The computer environment . . . should be providing the

facilitating structure and tools that enable students to make maximum use of their own

intelligence and knowledge. (p. 54)


This concept of procedural facilitation underpins Scardamalia and Bereiter’s previous work on

written composition, Palincsar and Brown’s (1984) “Reciprocal Teaching,” Brown and

Palincsar’s (1989) “Guided, Cooperative Learning” approach, and Collins, Brown and

Newman’s (1989) “Cognitive Apprenticeship.” Concepts of procedural facilitation have been

expanded and exist in current forms as scaffolds in Knowledge Forum and strategies for

collective Knowledge Building. A series of design principles for CSILE grew out of these initial

experiments. They include:

· Make knowledge-construction activities overt

· Maintain attention to cognitive goals (rather than task goals)

· Identify knowledge “lacks” and provide positive consequences

· Provide process-relevant feedback (as opposed to product feedback)

· Encourage learning strategies as opposed to rehearsal/memorization

· strategies (i.e. drill and practice approaches)

· Encourage recursive, multiple-pass strategies through information

· Support varied ways of knowledge organization and representation

· Encourage unlimited use and examination of existing knowledge

· Provide opportunities for reflectivity and individual learning styles

· Facilitate transfer of knowledge across disciplinary contexts and

· curriculum boundaries

· Give students more responsibility for contributing to each other’s learning

(Scardamalia et al., 1989)

Scardamalia et al. (1989) indicated that identified knowledge “lacks” should be met with positive

consequences, possibly from program feedback, resulting in enhanced possibilities for

overcoming these lacks and fully achieving the learning goals. Current notions of embedded and

transformative assessment (Scardamalia, 2002) can be traced back to these early roots. Many of

the concepts articulated are pertinent to current educational contexts and the context of the study

herein.

In Bereiter and Scardamalia’s (1990) paper, “An Architecture for Collaborative Knowledge

Building,” the authors described the pivotal concept of community, and the importance of a

Knowledge Building community as opposed to individual knowledge work. They indicated that

the guiding idea for design and application of CSILE was to support Knowledge Building

communities, which they likened to successful scientific research groups. Influential educational

concepts of constructivism, “the idea that knowledge is a human construction, not something that

exits to be revealed and transmitted”; sociocultural activity, “as the medium through which

knowledge construction takes place”; and apprenticeship, “in that much of the skill of the young


scientist is acquired by working along with a more mature scientist” were described (Bereiter &

Scardamalia, 1990, p. 42).

Bereiter and Scardamalia’s (1990) continued evolution of design principles can be summarized

as follows:

1. Objectification. Reference is made to Carl Popper’s concept of the objectification of

knowledge. Knowledge should be treated as an object outside the mind to be criticized,

modified, related and compared from different viewpoints in different contexts. Scardamalia and

Bereiter (1990) stated:

In this way the danger of knowledge being treated as an absolute, devoid of human bias,

can be minimized. Community critique, examination and belief revision are central to

knowledge work. The problem in conventional classrooms is that knowledge is generally

transmitted as an absolute and critique is subordinated to tasks and activities. (p. 44)

2. Progress. Knowledge Building should evoke what Whitehead called “disciplined progress”

that is evident to participants. Much of learning in conventions situations is cyclical rather than

progressive.

3. Synthesis. CSILE should support continual synthesis and progressive construction of higher

order knowledge representations and understandings. This process should be formative rather

than summative. CSILE should counteract the tendency of hypermedia to create a loosely

connected network of ideas lacking organization, synthesis and progress.

4. Consequence. Positive reinforcement as a result of knowledge processing operations is

advocated. In a scientific community reinforcements include citation, comment on, use and

confirmation of one’s ideas by another. Such reinforcements could be provided in CSILE. Ideas

advancement can be perceived by the entire community in CSILE.

5. Contribution. Distributed discourse is intended to enhance depth of understanding through

multiple perspectives and to raise the knowledge of the collective, not just the individual.

Development of expertise is defined by a progressive problem solving process; problems are

continually reformulated at increasing levels of complexity (Bereiter & Scardamalia, 1990, p.

45).


These concepts, particularly communal development of expertise, stand in contrast to the

development of individual expertise defined in terms of a staged, five-step, novice-to-expert

model (Dreyfus & Dreyfus, 1986). For Bereiter and Scardamalia (1993), the problem with such a

staged model is that expertise is defined as an end state. Instead, knowledge building reframes

development of expertise using a process model of expertise, characterized by reinvestment of

mental resources at higher and higher levels in a collaborative and progressive problem-solving

process (Bereiter & Scardamalia, 1993a; Scardamalia, 2002).

Scardamalia and Bereiter’s notion of intentionality remains key to concepts of continual

improvability. Intentionality has evolved into the robust principle of Knowledge Building known

as epistemic agency (Scardamalia, 2002). Examination of the early definitions of intentional

learning are informative and still relevant. Scardamalia and Bereiter (1989) indicated:

We use the term intentional learning to refer to cognitive process that have learning as a

goal rather than an incidental outcome. . . . Whether intentional learning occurs is likely

to depend on both situational and intrinsic factors - of what the situation afford in goal

attainment opportunities and on what the student’s mental resources are for attaining

those goals. Thus, focusing on intentional learning provides a natural way of coordinating

the two relevant research traditions . . . learning situations and . . . learning skills. (p. 363)

Bereiter and Scardamalia (1989) distinguished between learning through problem solving and

learning as problem solving. Learning thorough problem solving results from cognitive

operations applied to knowledge states in an effort to advance knowledge toward the goal. In

contrast, learning as problem solving implies that the goal itself is a learning goal. Learning

through problem solving is aimed at acquiring a deeper understanding and competence for

purposes of lifelong learning (Bereiter & Scardamalia, 1989, pp. 365–366). Learning as problem

solving is aimed at completing learning tasks and activities:

Metaknowledge, or knowledge about knowledge and personal assessment and

understanding of one’s own knowledge and skills is needed for intentional learning. The

three major metaknowledge needs are characterized as: 1. A problem solving framework

for approaching learning, that is an “executive structure”; 2. Awareness of the functional

potential of knowledge, that is that knowledge can be “transformative.” Rather than

considering knowledge as a new fact to be added to a collection, it can set off a reflective

process to change a central belief, to connect facts to build new understanding; 3.

Strategies for identifying deficits in one’s own knowledge is a vital part of intentional

learning. Without it, the only kind of learning goal one can set is to learn more about a

topic. ...When learning is approached within a problem solving framework, identifying

what one does not know becomes a variety of problem-finding. (Bereiter & Scardamalia,

1989, pp. 375–377)


Identification of problems is considered to be “an active, strategically guided process,” not a

simple whim of curiosity.

The conclusion that Bereiter and Scardamalia (1989) arrive at is:

In order to learn what is ostensibly being taught in school, students need to direct mental

effort to goals over and above those implicit in the school activities. Without such

intentional learning, education degenerates into the doing of schoolwork and other

activities. (p. 385)

This cautionary statement applied to education of youth can also be applied to that of adults and

the context of continuing medical education.

The pursuit of cognitive goals, over and above the requirements of tasks is what Bereiter and

Scardamalia have called intentional learning. Strategies to empower students to pursue cognitive

goals can be met by teachers teaching students about metaknowledge; teachers progressively

turning over high level functions (e.g., reframing questions or summarizing) to students; teachers

modelling the process of setting level, more meaningful cognitive goals beyond schoolwork.

Higher-level cognitive goals require self-assessment of level of constructive effort since it cannot

be judged as a task.

Scardamalia and Bereiter’s (1989) framing of all levels of education as a Knowledge Building

community shifts responsibility, in the sense that there exists not only a body of knowledge to be

acquired, but a dedication towards contribution and advancement to the body of knowledge, over

and beyond individual achievement. This framing is fitting for current examination and

reconceptualization of continuing medical education, and addresses potential physician

commitment to not only to lifelong learning, but to pervasive Knowledge Building.

2.3.4 Belief Mode and Design Mode

“We do not suggest that ‘design mode’ should replace ‘belief mode’ in schools. Both are

important. . . . Knowledge creation depends on moving back and forth between belief and design

questions in ways that maintain progress in idea improvement” (Bereiter & Scardamalia, 2007,

pp. 25–26). These concepts are complex and demand further explanation.

Knowledge Building has been characterized as sociocognitive “work at the edge of knowledge”

(Scardamalia, 2003b). In order for work to occur at the current edge of knowledge, students and


communities must have an up-to-date and deep of understanding of the belief-mode edge of

ideas and artifacts. Students must know what the facts and larger concepts are, how they are

currently defined, what the evidence is that supports these ideas, how these ideas stand in

relation to one another—so that they can intentionally ask both “good” design mode and belief

mode questions (Bereiter & Scardamalia, 2003, 2007). Bereiter and Scardamalia (2007) indicate:

Good questions to ask in belief mode are:

· What does this statement mean?

· Is it true?

· What’s the evidence?

· What are the arguments for and against?

Important questions to ask in design mode are:

· What is this idea good for?

· What does it do and fail to do?

· Does it have a future?

· How could it be improved? (pp. 25-26)

Work in belief mode and design mode is important in continuing medical education. Belief mode

work emphasizes truth, evidence, and personal knowledge. Doctors deal with belief issues in

terms of assessing information quality, which allows for consideration both of degree of certainty

and of the consequences of being wrong. They need to know the most current science, facts and

evidence, treatment and management to perform best-practice care. Design mode work deals

with issues of usefulness, adequacy, fruitfulness, and improvability (Bereiter & Scardamalia,

2003, 2007). The objectives are idea advancement and knowledge creation. Continuing medical

education needs to also address improvement in personal and community practice, locally and

world-wide. Design mode work requires a different type of thinking around different questions

than belief mode work. The problem space must enable both modes and ideally create synergy

between them.

2.3.5 The Knowledge Building Problem Space

Open and ill-defined problem spaces provide opportunities for emergence and self-organization

(Scardamalia & Bereiter, 2006) and represent distinguishing features of design-mode

environments. A technically unstructured problem space such as Knowledge Forum can provide

collective opportunities for testing understandings, identifying issues, and advancing concerns,

and can also provide challenges for introducing structure to problematize an issue or clinical case

contributed by participants, the facilitator, or the curriculum designer.


The Knowledge Building/Knowledge Forum problem space in this study is both structured and

unstructured. It is structured by a clinical case and learning objectives that presuppose discourse

structures of case-based reasoning (Kolodner, 1992) and problem-centred knowledge (Bereiter,

Burtis, & Scardamalia, 1994); it is also unstructured, in that the case is used as a springboard,

intended to open discussion, not close it down.

Structures vary across contexts, communities, and cultures and are determined by participant

interactions. Facilitator/student participant structures that are more symmetric have been found to

shift power and responsibility to students resulting in greater agency and authority (Tabak &

Baumgartner, 2004).

Knowledge Building student-teacher participant structures have not received much attention in

the research literature. Important studies in a grade-school context in Knowledge Forum

provided interesting results on teacher-student question and exchange patterns (Zhang,

Scardamalia, Reeve, & Messina, 2009; Zhang, Scardamalia, Lamon, Messina, & Reeve, 2007,

Zhang et al., 2005). Typical patterns like those found by Cazden (1988) characterized by teacher-

initiated questions followed by student responses were found, as well as those that went beyond

this structure by building on students’ ideas. Details on studies involving participant structures

and Knowledge Building facilitation will be provided later in this review of the literature. These

studies were used to inform ideas and methods for this research study.

2.3.6. Participant Structures and the Codesign of Knowledge Building

2.3.6.1 Facilitating Knowledge Building. Ng & Law (2003) found that a facilitator’s ability to

scaffold online Knowledge Building was particularly important when the subject matter context

was beyond the students’ zone of proximal development. In another study with young students,

Viilo, Seitamaa-Hakkarainen, and Hakkarainen (2011) analyzed a teacher’s weekly reflections

on facilitating knowledge building online and indicated that “the essential question, when

analyzing teacher’s support for collaborative inquiry project, is who is carrying the responsibility

for advancement of the process and who is posing the questions to be asked” (p. 51). They found

the teacher had a crucial influence in the process and recommended further research to cultivate

models and ideas of social cultural practices of knowledge building facilitation and how these

practices are transformed by technology.


Hmelo-Silver and Barrows (2008) examined face-to-face facilitation of knowledge building with

second-year medical students in a problem-based learning group. They found that the facilitator

supported knowledge building by “asking open-ended metacognitive questions and catalyzing

group progress. The students took responsibility for advancing the group’s understanding as they

asked many high-level questions and built on each other’s thinking to construct collaborative

explanations.” Hmelo-Silver and Barrows explained that helping facilitators and students ask the

right kind of questions to provoke building on each other’s thinking may be a key to

orchestrating knowledge building discourse.

Based on the strength and findings of these studies, the study herein will examine social network

structural and semantic analysis primarily of build-on notes and focus on the relationship of

questions to participant structures and Knowledge Building.

2.3.6.2 Facilitating knowledge building in relationship to coaching reflection-in-action.

Similarities exist between the descriptions of knowledge building facilitation in an undergraduate

medical education problem-based learning seminar (Hmelo-Silver & Barrows, 2008) and the role

of professor as coach in an architectural design studio setting (Schön, 1983). The dialogue

between coach and student and the paradoxes of learning to design highlight relationships

between reflection-in-action and knowledge building. These relationships are worth considering;

however, the key feature of knowledge building as a social cultural practice remains unaddressed

in Schon notion of reflection-in-action, which he describes as dialogical, between coach and

student and personal (Schön, 1983, 1987).

Schön (1987) focused on the need for artistry in professional education and for future practice;

his work has been influential in continuing medical education: “The studio tradition of design

education is consistent with an older and broader tradition of educational thought and practice,

according to which the most important things—artistry, wisdom, virtue—can only be learned for

oneself” (p. 84). He advocated for participatory, reflective practice.

Schön (1987) indicated the knowledge that one needs to know “will not come from teaching but

from questioning” ( p. 85). Like the knowledge building studies referred to previously,

questioning is seen as central to the process of facilitating improvement.


Schön (1987) reframed teaching, quoting Carl Rogers, who believed that “the most important

things cannot be taught but must be discovered and appropriated for oneself. . . . [One] attributes

to [oneself] and others a capacity for self-discovery and functions as a paradoxical teacher”

giving central importance to one’s own role as a learner (p. 92). The importance of Knowledge

Building symmetry and relationship to agency has been discussed.

Overcoming the “paradoxes and predicaments in learning to design” depends on the capacity of

student and studio master to communicate effectively with each other, in spite of the potential for

vagueness, ambiguity, or obscurity inherent in the things about which they try to communicate.

Their search for convergence of meaning,” as in knowledge building, is what is at stake (Schön,

1987, p. 99).

Schön called the process of designing a “reflective conversation with the situation.” A brief

review here of the designer’s personal reflection-in-action sets the stage for how the dialogue

between the coach and student can further reflective processes. Here again we note two levels:

the designer’s reflection with the situation, and with the coach’s reflections on the designer’s

reflections. In this way, the role of teacher becomes that of colearner. Reflection on situation is

active:

[The designer]. . . works in particular situation, uses particular materials, and employs a

distinctive medium and language. Typically, his making process is complex. There are

more variables - kinds of possible moves, norms, and interrelationships of these - than

can be represented in a finite model. Because of this complexity, the designer’s moves

tend, happily or unhappily, to produce consequences other than those intended. When this

happens, the designer may take account of the unintended changes he has made in the

situation by forming new appreciations and understandings and by making new moves.

He shapes the situation, in accordance with his initial appreciation of it, the situation

“talks back,” and he responds to the situation’s back-talk.

In a good process of design, this conversation with the situation is reflective. In

answer to the situation’s back-talk, the designer reflects-in-action on the construction of

the problem, the strategies of action, or the model of the phenomena, which have been

implicit in his moves. (Schön, 1983, p. 79)

“Three dimensions of this process are particularly noteworthy: the domains of language in which

the designer describes and appreciates the consequences of his moves, the implications he

discovers and follows, and his changing stance toward the situation with which he converses”

(Schön, 1983, p. 95).


The implications of the designer’s moves come at “choice-points” (Schön, 1983, p. 99). The

designer considers not only the present choice but also a tree of further choices. After

consideration of many of a few “what if” situations the designer must make a binding choice that

has implications for all further moves. Thus there is a continually evolving web of implications

within which the designer reflects-in-action. As the designer spins out his web of moves, a

continual shift in stance occurs. The designer must oscillate between the unit and the total,

between involvement and detachment.

Now that we have considered the designer’s personal reflection-in-action with the situation

encountered with the artwork, we now consider how the dialogue between the coach and student

can further this process. “Questioning, answering, advising, listening, demonstrating, observing,

imitating, criticizing—all are chained together so that one intervention or response can trigger or

build on another” (Schön, 1987, p. 114). The important notion of building on ideas, as in

knowledge building, is evident here. Schön (1987) likened the chain of reciprocal reactions and

reflections of the student and coach to a ladder where the “higher levels of activity are “meta” to

those below. The levels of action and reflection on action can be seen as the rungs of a ladder.

Climbing up the ladder, one makes what has happened at the rung below into an object of

reflection” (Schön, 1987, p. 114).

It is noteworthy that facilitation of cognitive design work in the art studio is similar in some

aspects to facilitating knowledge building. However, there are three key differences: knowledge

building social cultural practices focus on collective advancement as opposed to teacher-student

dyads; knowledge work is intentional which includes reflection early in the process; and artifacts

for knowledge work are objectified. This latter includes reflections, which in Schön’s world

often remained located within one’s mind, not objectified for further work.

Similar to Schön’s (1983) articulation of four steps of reflection, Mamede and Schmidt’s (2004)

five-factor model of reflective practice in medicine consisted of “deliberate induction; deliberate

deduction; testing and synthesizing; openness for reflection; and metareasoning.” Mamede and

Schmidt’s framework advocated a meta level that does not seem attainable in isolated individual

work to the same degree as community knowledge building.


2.3.6.3 Expertise and social networks for cocreation. Bereiter and Scardamalia (1993)

addressed the issue of the development of expertise in their book, Surpassing Ourselves: An

Inquiry Into the Nature and Implications of Expertise. They distinguished between “expert-like

and non-expert-like learners” and defined expert-like learners as those who are engaged in

progressive problem-solving through distributed knowledge building, and working on communal

artifacts.

Regehr and Norman (1996) stated that expertise is characterized by the development of elaborate

semantic networks and that true understanding is not defined by the quantity of information a

person possesses but by the extent to which this information is organized into a coherent,

mutually supportive network of concepts and examples. The semantic network is viewed as an

elaborate set of connections between abstract concepts and/or specific experiences, and the links

are viewed as linkages between concepts and experiences that are based on meaning. The issue

here once again is that this web of knowledge is located inside one’s mind.

Yet the notion that meaning and understanding is more than memorization of facts is highly

relevant to continuing medical education. Meaning is enhanced through activation of relevant

prior knowledge that facilitates processing of new information and problem discussion stimulates

activation and elaboration of prior knowledge that subsequently facilitates processing and

comprehension of new information and retrieval (Norman & Schmidt, 1992; Regehr & Norman,

1996; Schmidt, Norman, & Boshuizen, 1990). Importantly for this study, Regehr and Norman

(1996) indicated that elaboration is the process of considering a piece of knowledge in a richer,

wider context. Strategies of elaboration are discussion, writing, answering questions, or using the

knowledge to understand a problem (Norman & Schmidt, 1992; Regehr & Norman, 1996).

It is evident how Regehr and Norman’s (1996) more traditional cognitive-psychology research

on personal semantic networks, within one’s mind, may contribute to an understanding of larger-

community. What is missing is the importance of objectified knowledge work aimed at

intentionally advancing ideas of importance to the broader community, the social network. This

is Knowledge Building collective responsibility of professional practice and expertise. This

distinguishing feature of Knowledge Building can be described using social network analyses

and tools in Knowledge Forum.


2.3.7 Knowledge Forum Suite of Analytic and Social Network Tools to Support

Knowledge Building

Knowledge Forum is second-generation CSILE. It incorporates social network and semantic

analysis tools. Using social networks to analyze patterns is a common research and evaluation

strategy, and has a history in the learning sciences, and particularly within the knowledge

building research community (Hewitt; 2005; Hewitt & Brett, 2007; Hewitt & Teplovs, 1999;

Zhang et al., 2005).

Salomon (1994a, 1994b) and Chi (1997) used mapping techniques to analyze difference in

patterns in participation and interaction. Chi (1997) presented a “practical guide” on how to

quantify qualitative analysis. She indicated that by mapping the data, one can begin to seek

patterns in the results, and these patterns can be statistically confirmed. In a study of children and

dinosaurs, Chi and Koeske (1983) found a correlation between density of links in a cluster and

better organization and coherence ideas, when compared to a more diffuse cluster structure, even

though the total number of links and nodes used to represent the two structures were identical

(Chi, 1997, p. 299).

Chi (1997) said:

In general, it does not appear difficult to quantify the density of graphical representations

of coded data . . . one should not have to rely on subjective visual assessments of the

structure of graphical representations. . . . Quantifying a perceived pattern in a graphic

representation of coded data is not the same thing as quantifying the total number of

elements in it. (p. 300)

It is important to note that the quantification occurred at the level of if the pattern, which

attempts to capture the structures in the representation.

Interpretation of perceived patterns in data and checking the validity of interpretation is key. Chi

(1997) noted interpretation depends entirely on the hypotheses being tested, the research

questions asked and the theoretical orientation of the investigator. Interpretation of data can be

performed in terms of strategies and processes, or structure and content of knowledge or both.

She cautions that the interpretation of patterns in data is often more persuasive if there are other

converging evidence or analyses.


Knowledge Building researchers not only looked at patterns of participation, but advanced this

idea by adding other dimensions, such as examining growth in conference threads (Hewitt &

Teplovs, 1999), why threads of discourse die in asynchronous environments (Hewitt, 2005), and

relationships between number of participants and online activity patterns (Hewitt & Brett, 2007).

Early notions of social network analysis were embedded in the Knowledge Forum analytic

toolkit (ATK; Burtis, 2001). The ATK could almost instantly calculate who built on whose notes

in a Knowledge Forum view and provide a numeric grid of results. The names of all participants

where listed on the x and y axes and a matrix of numbers was presented indicating the number of

times one person built on another’s note or notes. This visual representation was difficult to

make sense of and did not easily facilitate pattern recognition. Even more perplexing and

challenging were comparisons across time. Limitations in representation of data and difficulty of

interpretation led this researcher not to include these data in this study.

New Knowledge Forum Social Network Visualization tools (Teplovs, 2010) enable us to see the

network of connections, analyze the relationships between connections, and over time see shifts

is patterns, that may signify shifts in knowledge building or belief mode to design mode

thinking. In Teplovs’s (2010) doctoral thesis study he used notes as a proxy for ideas. In essence

the network visualization was not just of connected notes, but a network of ideas. He stated that

although notes do not correspond exactly with idea, most notes contain at least one or more

ideas, and therefore are a convenient unit of analysis. Teplovs created a systems architecture, in

particular a visualization layer that he describes lies above the discourse layer. The “Knowledge

Space Visualizer (KSV) . . . is a prototypic system for showing reconstructed representations of

discourse-based artifacts and facilitating assessment in light of patterns of interactivity of

participants and their ideas” (Teplovs, 2010, p. 1). The Knowledge Space Visualizer “uses Latent

Semantic Analysis to extend techniques from social network analyses, making it possible to infer

relationships among note contents,” (Teplovs, 2010, p. 1), thereby enabling the study of idea

networks in conjunction with social networks in online discourse.

The Knowledge Space Visualizer provided quantitative network metrics of density and degree,

along with the visualizations. It provided new ways to study relationships between notes, to

examine changes over time (not improvability of ideas), and to support assessment—for

embedded, concurrent feedback (as in Teplovs’s thesis) and for research-based innovation. This


study makes use of the Knowledge Forum network analytic tools of density and degree. These

new tools were important to the current study.

A number of recent conference papers and workshops have disseminated new ideas about

visualization assessments of Knowledge Building that are highly relevant to advance research.

These include identification of idea clusters (Teplovs & Fujita, 2009), use of latent semantic

analysis and term clouds (Teplovs & Fujita, 2009), pinpointing of pivotal moments (Fujita &

Teplovs, 2009), and visualization of group cognition using semantic network analysis (Sha,

Teplovs, & van Aalst, 2010). These types of advances are key to developing new tools to assess

Knowledge Building and collective innovation. The next section will explore the use of social

network analysis.

2.4 Cluster 3: Social Network Measures and Sociocognitive Dynamics

That Enable Work Over and Above Traditional Learning

Work beyond best practices requires work beyond traditional assessment, which in turn requires

new tools to answer new questions and investigate new perspectives. In this section I review

literature on network measures and sociocognitive dynamics to explore knowledge work focused

on collective outcomes and over and above traditional learning.

2.4.1 Social Network Analysis in the Social Sciences

Engeström, Miettinen, and Punamäki (1999) provided a framework in which to understand

activity theory from both individual and social perspectives, as a system mediated by artifacts

through “expansive” cycles of internalization and externalization for transformation. Engeström

et al. (1999) indicated that “Key findings and outcomes of such research are novel activity-

specific, intermediate-level theoretical concepts and methods—intellectual tools for reflective

mastery of practice” (p. 36). The concept of examining activity as a principle of explanation is

aimed at identifying potential for qualitative change at the local level of activity, as well as at a

grander scale:

The mightiest, most impersonal societal structures can be seen as consisting of local

activities carried out by concrete human beings with the help of mediating artifacts even

if they may take place in high political offices and corporate boardrooms instead of

factory floors and street corners. In this sense, it might be useful to try to look at the

society more as a (layered network of interconnected activity systems and less as a


pyramid of rigid structures dependent on a single centre of power. (Engeström et al.,

1999, p. 36)

Engeström’s (1999) ideas have been highly influential in social psychology and educational

technology research. However, the roots of social network analysis are often attributed to Linton

C. Freeman (1978/1979, 2004), a social scientist and mathematician.

Freeman’s (1978/79) analysis of structural centrality in social networks provided us with three

measures: one absolute; one relative, a measure of centrality of position in a network; and one

reflecting the degree of centralization of the entire network. Graph theory and point measures

were calculated at 34 levels of centrality. Overall, Freeman’s network centrality measures

indicated that the star or wheel pattern is assigned the maximum centrality score and the circle

the minimum score: “between these extremes . . . we are faced with an embarrassment of

intellectual riches. We have not one, but three conceptions of centrality, and we have a family of

measures for each” (Freeman, 1978/79). Thus centrality can affect group processes and be

interpreted in three ways. For example, if we propose that leadership depends on centrality, then

we need to know if this means centrality as control, centrality as independence, or centrality as

activity. Any one or a combination of them may be appropriate in an application such that within

this study.

Other social network scientists have contributed to ideas, measurements, and interpretations of

these measures more recently. Barry Wellman (1983) has done much to clarify basic principles

of network analysis. He has reframed our thinking in this field by pointing out that the world is

composed of networks, not groups (Wellman, 1988). Marin and Wellman’s (2011) recently

published introduction to social network analysis is particularly helpful in setting forth guiding

principles of network analysis that direct our attention to the fact that this is research based on

relations, not attributes.

Social network analysis has flourished in the social sciences in recent years and consequently

many books and articles have been published illuminating methods of data collection,

mathematical representations of social network, structural properties, such as

centrality/prestige/prominence, group cohesion (including cores and cliques), positional analysis

(including roles and ties between and within positions), and block measures, such as network

density (Carrington, Scott, & Wasserman, 2005; Scott, 2009;Wasserman & Faust, 1994).


Even more recently, along with publications on social network structural analysis methods, came

publication of more conceptual works and results of research studies. Current issues, methods,

and a breadth of research reports from many different societal aspects are covered in the 2011

publication of The Sage Handbook of Social Network Analysis (Scott & Carrington, 2011).

However, this extensive volume does not include studies related to health care, medicine, or any

aspect of medical education. Another recent publication, Social Networks and Health: Models,

Methods, and Applications (Valente, 2010), fills this gap and specifically targets the heath care

audience, but not health sciences education. Despite this, helpful connections are made between

network analysis outcomes and applications to identification of opinion leaders, key players,

bridge participants and group characteristics important to health care.

In a more conceptual vein, power structures of social networks are explored by John Scott

(2001), in his book entitled Power. Scott’s writing was helpful in framing this research study,

particularly in terms of his work on expertise and professionalism and authoritative structures; he

links his ideas to Foucault previous work and establishes ways of interpreting power analyses.

Social network power relationships have also been analyzed within the classroom; one

particularly important study is reported in the next section. Other sections will review results of

studies pertaining to various aspects of social networks within different environments, face-to-

face and online.

2.4.2 Social Network Analysis and Power Structures in Education

Four important studies are reviewed in the next section. They are separated from others below in

the education section, since they were highly influential in the conceptual development and

methodological design of this study.

2.4.2.1 Power structures concepts. Cornelius and Herrenkohl (2004) used interviews to

examine power structures in Grades 5 and 6 classrooms and found that with the introduction of

new technology, new participant structures emerged, changing relationships between students

and the teachers, amongst students themselves, and their relationship with the learning concepts.

Social network analysis of changes in social interactions and participant structures clarified “the

way that these structures enable and incite disciplinary thinking.” The emergence of new

participant structures transforms power and authority:


Further we believe that looking at participant structures in terms of power allows an

effective means for analyzing why some structures are more successful than others, whey

certain students may “appropriate” the structures more readily than others, and why some

structures may be better suited to certain disciplines than others. (Cornelius &

Herrenkohl, 2004, p. 468)

The authors framed participant structures within a notion of cultural tools and explained that by

doing so it was their intention to move beyond behavioural explanations of the individual, and to

focus on relationships.

Similarly, Cornelius and Herrenkohl (2004) framed the concept of power in terms of Foucault’s

(2001) definition; the nature of power is seen as ‘“strictly relational” and as containing “many

points of resistance”(p. 477). In an educational setting, this means that manifestations of power

could be found in any interaction or relationship. Power is not seen as external to learning, nor is

it seen as imposed from the top down; it is seen as inherent to groups and individuals. Power is

not seen as stable; it is seen as dynamic, continually changing. In Cornelius and Herrenkohl’s

(2004) conceptualization, “relationships of power as existing on a balance scale, with situation

factors causing the positions of persons in an environment to constantly shift and change with the

potential of being tipped in different directions” (p. 469). A sociocultural approach to the

examination of power is taken in this study. They indicate a new vocabulary is required to

explicate power relationships located in interactions and for conceptualizing dynamic ways in

which people, mediating artifacts, and sociocultural tools, such as new participant structures

influence each other.

Results of the current study indicate that three conceptualizations of power relate to classrooms

and education: ownership of ideas, whether that be student or teacher; partisanship, which

describes relationships among students and their interaction with concepts and with other; and

persuasive discourse, which relates to ways in which ideas are communicated, as authoritative or

internally persuasive (Cornelius & Herrenkohl, 2004). Additionally, each of these

conceptualizations reflects different and corresponding relationships that exist in the classrooms:

between students and concepts; among students; and between students and teachers. Results

indicated ownership of ideas narrowed the distance between students and the scientific concepts;

students created their own theories and used ideas flexibly. Partisanship enabled them to

collectively explain ideas; and, persuasive discourse allowed them to try to convince their peers

by providing proof for their arguments to support their theories.


Cornelius and Herrenkohl’s study (2004) also indicated that the teacher’s role was central to

students’ accomplishment through four main structures: (a) problematizing content, through

building-on prior knowledge; (b) questioning, and challenging ideas; (c) giving students

authority, recognizing students’ ideas and contributions, and holding students accountable; and

(d) providing relevant resources. Discourse sequences were found to be very different from

Cazden’s (1988) I-R-E sequence, in which the teacher initiates most questions, students respond,

and then the teacher evaluates. Teachers play an important role in shaping student participation,

by highly scaffolding interactions and creating opportunistic power relations (Cornelius &

Herrenkohl, 2004). The lens of power can provide new perspectives and insights into the

strengths and weaknesses of participant structures in the classroom. This analytic lens also

allows for new explanations of the social networks in education, such as how students participate

or collaborate. Position analysis of power is used in the current study.

2.4.2.2 Social network analysis of roles and power structures. In another study, Aviv,

Erlich, Ravid, and Geva (2003) identified roles and power structures through social network

analysis of cohesion and cliques. They used mixed-methods, combining social network analysis

with content analysis, to examine two asynchronous learning environments, in two university-

level business ethics courses. One was structured and a formal part of a course and the other was

considered unstructured and not a required part of the course. Analysis of power or centrality

was measured by Eigenvector centrality, and Freeman’s in-degree and out-degree centrality. In

additional network density was calculated and compared. They found that the structured design

was associated with a high degree of knowledge construction. Many cohesive cliques developed.

Students took on bridging and triggering roles and the tutor held relatively little power, but

closely monitored the discourse. Students belonging to more than one clique often perform

bridging roles in that they facilitate the flow of information. In the non-structured environment

activity was lower; there were fewer cliques and most of the students were passive, following the

teacher who held a central position of power. Social network measures used in this study were

used in the current study.

2.4.2.3 Social network position and identity. Ligorio (2009) explored the relationship

between knowledge building and identity building. She indicated, in contrast to Lave and

Wenger (1991), that identity in a community of practice (Wenger, 1998) is not always developed

through a linear movement from the periphery to the centre; Ligorio found that, in fact, this


process could described by three trajectories in addition to Lave and Wenger’s original one.

These four trajectories are relevant to the current study:

Stability. Some students tend to maintain the same level of centrality over time.

Progressive centralization. This is a linear trajectory, from the periphery toward

increasing centrality, as described by Lave and Wenger (1991).

Progressive decentralization. This indicates an inverse linear trajectory to that above,

towards an ever-diminishing centrality.

Non-linear stability. This characterizes the trajectory of decentralization in the middle of

the discussion, before adopting a central position at the end of the discussion; or,

conversely, central position in the middle of the discussion, before decentralizing.

(Ligorio, 2009, p. 40)

Ligorio (2009) concluded that these different trajectories influence the structure of the online

community and that students who develop a stabilized trajectory of centrality become a

“referential nucleus” within the community for others to build around (p. 40). Ligorio (2009,

2010) put forward the notion of “digital I-positioning” aimed at personal change and indicates

that a “community is transformed on the basis of the interactions between positions that take

place simultaneously on multiple levels” (Ligorio, 2009, p. 41), including the individual,

interpersonal, and community levels. Development of self and sense-making through

examination I-positioning sheds interesting new light relevant to professional development along

trajectories of lifelong learning.

2.4.2.4 Social network analysis of facilitator roles and participant structures. Tabak and

Baumgartner (2004) explored teacher-student participant structures focusing on analysis of the

teacher in achieving balance between authoritative and persuasive discourse with science

students (face-to-face in a classroom). They demonstrated how the mastery of the cultural tool

occurred through symmetric student-teacher interactions in a participant structure they called

partner. Although it was rare in comparison to other observed structures, mentor and monitor,

they determined that symmetrical partner structures can provide profitable opportunities for

knowledge work. They indicated that “the partner participant structure is similar to the mentor

participant structure, but the main difference lies in the posture that the teacher assumes (Tabak

& Baumgartner, 2004). In the partner structure, the teacher and students are said to share the

same role of investigator; in the mentor structure the teacher is an outsider and scaffolds the

students to align their discourse with the norms of study. Tabak and Baumgartner concluded:


“Rather than representing opposing or competing instructional forms, the mentor and partner are

efficacious as an ensemble. . . . Balance is achieved between authoritative and persuasive

discourse” (p. 424). The notion of teacher-as-partner, as opposed to coach or guide-on- the-side,

holds particular promise. Tabak and Baumgartner suggested that it is possible to promote

rigorous subject-matter learning typically attributed to formal learning environments as well as

the symmetry and personal outcomes typically attributed to informal learning environments. I

conducted qualitative, within-note analysis of participant structures using the modified categories

from the Tabak and Baumgartner study. My study found a similar efficacious ensemble in online

facilitation—balance between the mentor and partner and expert stance was readily apparent.

2.4.3 Social Network Analysis of Sociocognitive Dynamics in Knowledge Building

In the last 10 years, social network analysis has been conducted in different educational

environments; published results are interesting and varied and range from grade school students

to higher education. No published social network studies were found in the context of continuing

medical education. However, information on the Internet was found on two studies in process

with undergraduate medical students and one with physicians in practice.

2.4.3.1 Social network analysis of Knowledge Building in K-12 classrooms. Sha and van

Aalst (2003), Philip (2010), and Moss and Beatty (2010) have employed social network analysis

of discourse in Knowledge Forum.

An early study by Sha and van Aalst (2003) aimed to probe collaboration at a more systemic

level. In two case studies (one with a Grade 4 class and one with a Grade 9 class) they employed

Knowledge Forum analytic toolkit measures, such as Who’s read whose note?, Who’s built on

whom?, number of notes created, percentage of notes linked, and percentage of notes read; and

social network analytical measures of in-degree, out-degree, and betweenness. Although there

were no significant correlations between the social network analysis variables and the use of

Knowledge Forum features in this study, the methods are emphasized over the results. The point

being that meaningful and readily understandable collective assessment measures are required

for research and classroom assessment of socially constructed knowledge and understanding.

Individual activity measures cannot capture this aspect.


In another study by Philip (2010), student interaction patterns were examined in a Grade 5/6

Knowledge Building class. In this study Philip used a series of similar Knowledge Forum

analytic toolkit and social network analysis measures, as well as calculating network density.

Philip noted the relevance of Social Network density and centrality measures in combination

with more traditional analytic toolkit measures for new ways of revealing group processes of

collective knowledge building. Sociogram visualizations provide a quick tool for feedback on

collective participation.

Moss and Beatty (2010) used social network analysis visualizations and note content analysis to

demonstrate patterns consistent with the knowledge-building principle of democratizing

knowledge. Grade 4 students at all achievement levels participated equally and with agency

supported by an egalitarian online culture.

2.4.3.2 Social network analysis of knowledge building in higher education. Lipponen,

Rahikainen, Lallimo, and Hakkarainen (2003) used similar mixed-methods to analyze patterns of

asynchronous online participation and quality of discourse using content analysis in a course

with university level students. Quality of discourse was found to be relatively weak, although

participation and network density was strong. Thus measures of density can be misleading; there

are few comparatives to understand what is optimal density for high-quality learning. Numerous

postings of unrelated information can lead to information overload that doesn’t relate to

knowledge improvement. Therefore the missing component seems to be intentional knowledge

building or epistemic agency.

Some more recent studies have employed social network analysis for embedded concurrent

assessment of knowledge building (Wang & Li, 2007). Others have used social network position

analysis to determine how thought leaders, as core participants, interacted with their peers and

impact knowledge building (Waters, 2008). Most interestingly Lu Wang (2010) analyzed two

online university level courses in terms of core periphery structures and determined these

networks contained “structural holes” (Burt, 2004). Opinion leaders positioned at structural holes

exhibited different characteristics in terms of knowledge building from those positioned at the

core of the network.


The importance of structural holes and their relationship to emergent “good ideas” was first

described by Burt (2004) in the business literature. He said: “people who stand near the holes in

a social structure are at higher risk of having good ideas” (Burt, 2004, p. 349). He argued that

opinions and ideas within groups are more homogeneous than between groups. Therefore people

connected across groups, are potentially exposed to alternative ideas and different ways of

thinking, which in turn, affords greater possibilities of options and emergence of new ideas.

“New ideas emerge from selection and synthesis across the structural holes between groups.

Some fraction of those new ideas are good. . . . A good idea broadly will be understood to be one

that people praise and value” (Burt, 2004, p. 349). In essence, Burt indicated, people whose

networks spanned structural holes can become brokers of ideas and brokerage across structural

holes between groups is the mechanism by which social capital increases. This idea has

important implication for large-scale Knowledge Building studies with multiple interacting

groups.

2.4.3.3 Social network analysis of an international knowledge building network. An

important design research study examined the development and international spread of

Knowledge Building innovations through the Knowledge Society Network in Knowledge Forum

(Hong, Scardamalia, & Zhang, 2010). The Knowledge Society Network was examined in terms

of social interaction and idea interaction through analysis of contribution, shared problem space

and multiple problem spaces.

Cumulative growth (e.g., number of participants, readers), a series of matrix correlations based

on participant interaction patterns, and calculation of network degree of centralization (core

group, periphery group) are expressed both quantitatively and qualitatively for participant

interaction in a shared problem space. Idea interaction patterns, are also expressed quantitatively

and visually, through network of patterns of idea interaction of weak/strong-tie comparisons,

based on descriptions by Granovetter (1983) and Haythornthwaite (2002).

Hong et al. (2010) offered the following definition: “The degree of participation in SNA means

the total number of connections between a participant and other participants in a network; and

degree centralization is the mean degree number (Freeman, 1979).”


Although these quantitative and visual analyses successfully illustrated growth over time and

intensity (week/strong) participant and idea interaction, researchers indicated that these analyses

could not explain more subtle, metarelationships between participant and idea interaction— “for

instance, how ideas might interact with each other, within the weak, peripheral participation

network” (Hong et al., 2010, p. 8). The following four subnetworks were analyzed:

(a) Sustained Knowledge Innovation Network—with strong participant and idea

interaction; (b) Emerging Network—with weak participant and idea interaction; (c)

Intensive Participant Interaction Network—with strong participant interaction but weak

idea interaction; and (d) Frequent Idea Interaction Network—with weak participant

interaction but strong idea interaction. . . . Each subnetwork represents a type of network

dynamic, and it is interesting that all four of these dynamics are evident in KSN. (Hong et

al., 2010)

It is also interesting that the interpretation of these analyses can be considered to be assessment

of democratization of knowledge building participation, processes, interactions, and ideas.

Outcomes successfully identified strong participant and idea interaction, as well as isolated

subcommunities in the periphery and temporally distant Knowledge Forum views that can be

enhanced.

2.4.4 Social Network Analysis in Medicine and Medical Education

Social network analysis has been used in public health studies, research in medical education,

and in medical practice, to create information networks. Only one pilot study was found that

examined Knowledge Building in a continuing medical education environment (Punja, 2007).

No studies were found that focus on social networks relationships of Knowledge Building in a

continuing medical education course.

Social network analysis has brought to the fore new relationships important to public health

information. For example, Christakis and Fowler popularized the notion of social networks

through their studies on the spread of obesity (2007), smoking (2008), and happiness (Fowler

and Christakis, 2008). In another study, by Gardy and colleagues (2011), whole-genome

sequencing results were overlaid on social network analysis to trace a tuberculosis outbreak and

describe the outbreak dynamics at a higher resolution.

Recently, in undergraduate medical education research, Walton and Oswald (2011) examined

patterns of interaction in a second-year problem-based learning group at the University of


Alberta, Canada. They used Freeman’s normalized centralization measures to visually convey

differences between a highly interactive group and a less interactive group. They also used

graphical measures to convey percent of individual participation (verbal contributions to

discussion). In both groups they found that the tutor spoke more than 50% of the time. These

results could be used as formative feedback tools aimed at improving student responsibility and

contributions to the social network.

In another study, currently underway at the University of Ottawa’s Faculty of Medicine,

(Sutherland et al., 2011), social network analysis is being used to determine if is possible to

measure student competencies such as collaboration and other behaviours identified in the

CanMEDS roles. This longitudinal study will follow students from 2008 (first year) to 2012

(medical clerkship). An outcomes report is accessible online, as presented by Dr. Stephanie

Sutherland, at University of Ottawa, Academy for Innovation in Medical Education Rounds

(Sutherland, 2011). Characteristics such as network size, interconnectivity, and brokerage were

evaluated and reported to students.

Social network analysis has been used in the context of continuing medical education to help

identify and connect physicians with similar practice and personal characteristics with each other

to create informal and formal networks of learning, to facilitate information flow, and improve

accountability in health care organizations (Miller et al., 2008). In this vein, Keating, Ayanian,

Cleary, & Marsden (2007) used social network analysis to evaluate the spread of influential

discussions among primary care physicians in a hospital-based practice.

2.4.5 Complexity of Social Network Discourse: Visual Representation, Elaboration, and

Identification of Misconceptions

Knowledge Forum enables visualization of individual thinking and collective understanding.

Making visually explicit one’s knowledge, ideas, and understanding—or lack thereof, is a

foundational premise of Knowledge Building. The importance of working with “objectified

knowledge” (Bereiter, 2002c), putting “ideas-at-the centre” (Scardamalia, 1999), creating

“awareness of knowledge lacks” (Bereiter & Scardamalia, 1989) and misconceptions (Burtis,

Chan, Hewitt, Scardamalia, & Bereiter, 1993) have long been a part of the literature on

Knowledge Building.


“Making thinking visible” (Collins, Brown, & Holum, 1991, p. 1), making knowledge tacit

(Polyani, 1983) as a knowledge object (Popper, 1972) for collective design mode work is a

radical shift from traditional individual learning where cognitive processes, reasoning, and

misconceptions often remain invisible—in one’s head (Bereiter & Scardamalia, 1989). Some

misconceptions have been shown to be robust and are of general concern in education (Chi,

2005) and of particular concern in medical education (Lester & Tritter, 2001) and continuing

medical education (Hanna, Premi, & Turnbull, 2000). Misconceptions in online learning pose

additional concerns of propagation (Burtis et al., 1993). These issues are addressed through

collective as opposed to individual online learning. The benefits of making ideas explicit and

working collectively with knowledge become evident in this next series of studies.

Accuracy of clinical reasoning, identification of misconceptions, and correction of errors are key

concerns in at all levels of medical education and practice. Elstein (1999) examined heuristics

and biases as sources of error in clinical reasoning. Earlier work by Custers, Regehr, and Norman

(1996) reviewed mental representations of medical diagnosis, i.e. prototypical, instance-based,

and semantic networks, including schema and script models. The work by Bordage and

colleagues (1991,1994, 1997) on semantic networks is particularly relevant to the Knowledge

Building context of this study and lends further insight and a framework for analysis of

complexity of discourse that will be used herein.

Bordage (1994) specified and classified the organization of four types of clinical discourse:

reduced, dispersed, elaborated, and complied. He determined that elaborated structure is

associated with accurate resolution of complex problems, that is, 75% to 80% resolution for

elaborated discourses as opposed to near-zero resolution for dispersed discourses. Bordage noted

that by making the semantic connections more visible and conscious—for example, using them

when reading textbooks or during case presentations and discussions—students could learn to

solve a problem by defining it clearly first before blindly generating lists of diagnostic

impressions. Bordage distinguished between simply listing potential diagnosis, as opposed to

articulating differential diagnoses with appropriate justifications and elaboration on reasoning,

before at making a commitment to a working diagnosis. As the title of this 1994 paper indicates,

he argued for “Elaborated Knowledge: A Key to Successful Diagnostic Thinking.” Bordage,

Connell, Chang, Gecht, and Sinacore (1997) created and validated a framework for semantic

content analysis of clinical case presentations.


In a later study, Chang, Bordage, and Connell (1998) focused on problem representation, the first

step in problem solving. They found striking differences between successful and unsuccessful

diagnosticians both clinically and statistically. they indicated that successful diagnosticians had

more thorough and relevant problem representations than did the unsuccessful ones and did more

simultaneous comparing and contrasting of diagnoses. They acknowledged that differences in

knowledge or experience might explain success or unsuccessful diagnoses, however the key

finding was the differences in abilities to compare and contrast among several diagnoses

simultaneously by using relevant representations of the clinical problem. They state that in doing

so, the problem solver can construct a mental representation of the situation in a different

formats, using words, graphs, diagrams, tables, equations, and pictures. They further indicated

that using multiple formats for problem representation improved students’ problem-

representation skills and resolution. Chang et al. (1998) indicated that problem representation in

medicine could be viewed as abstraction of semantics; that is, words that “serve as verbal

representations of the problem that are useful in retrieving knowledge from memory, such as

visual representation of the pathophysiology or pathology (e.g., seeing in the mind’s eye an

inflamed joint with its various structures)” (p. S109). They indicated that abstraction of problem

representation in this way may correspond to greater diagnostic depth and accuracy, reflected in

elaborated or compiled discourses, as opposed to reduced or dispersed ones. Ironically, in many

medical schools, courses in physical diagnosis (which could promote abstract representations of

a patient’s problem) and in pathophysiology (which could promote abstract or visual

representations of disease) are typically taught concurrently but with little or no integration for

students. The findings from this study suggest that educational strategies should be used to

encourage integration of semantic and visual abstractions in problem representation to improve

diagnostic skill.

In the End-of-Life Care Distance Education Program, Knowledge Building in Knowledge Forum

provides opportunities for problem representation across various formats. Technically,

Knowledge Forum enables the translation of ideas and words to text, rendering them explicit, as

knowledge objects, in a visual format. Clinical videos have been used to provide visual context.

However, visual problem representation has not been used in the problem space, although

technically the opportunity exists. Hence, Knowledge Building through words and images in


open, problem space in Knowledge Forum can be used not only to enable visualization of

cognitive artifacts and processes, but also to represent problems.

A follow-up study demonstrated the positive outcomes of making clinical reasoning “more

visible and conscious,” as well as “who learns from whom” (Nendaz, Junod, Vu, & Bordage,

1998, p. S54). The use of a group management plan, called a metaplan, was created for students,

residents, and house staff to use in educational rounds. A short clinical vignette (in text) from a

real patient’s record was displayed on a pin board. Each participant, was given an anonymous

code number, was asked to write three ranked diagnostic hypotheses and a justification of each

hypothesis on a separate cards. Then the entire group was asked to pin up their cards (diagnosis

and justification side-by-side) and to group them by rank, on the board. Participants perused all

displayed card and then were allowed to post revisions. This rudimentary, pushpin display

method can be seen as a low-tech version of a structured exercise that could be more easily

conducted by creating Knowledge Forum notes for the visual display of cognitive artifacts in an

online problem space.

Cards were collected and scored by five physicians, using four categories: correct and complete

final diagnoses, correct but incomplete diagnoses, plausible diagnoses/part of the differential

diagnosis, or incorrect diagnoses. Justifications were categorized as essential findings,

contributing findings and non-relevant findings; misconceptions were noted as well. In terms of

discourse, it was found that experienced physicians used more compiled language; they used a

single summary word or set or terms (e.g., “Sjogren’s syndrome”) as opposed to more elaborated

descriptions of symptoms (e.g., “dry mouth” and “conjunctivitis”) to justify diagnoses.

Most importantly for our purposes were the findings on the effects of visual displays, from this

structured, group exercise. After visual display of the initial diagnostic hypotheses, subjects, with

less than eight years of training/practice, made on average two changes, such as adding or

deleting a diagnosis, changing the order of diagnoses, or refining an existing diagnosis. Subjects

with more than eight years of training made only one change. The addition of new diagnoses

were more frequent among fourth-year students than among physicians with more than four

years of experience. “Overall, more than three quarters of the diagnostic changes were beneficial

(95% CI = 73% to 82%), reaching 92% (95% CI = 83% to 97%) among fourth-year students (x2

(4) = 12.203, p = .01)” (Nendaz et al., 1998, p. S56). The visual displays were more beneficial to


subjects that initially listed an incomplete correct diagnosis, than to those who did not mention

the correct diagnosis initially. When a change had been made subjects were asked to note which

card had been most influential in their ideas for revision. Results indicated 70% of students relied

mainly on ideas from chief residents and residents (whose discourse has been noted to be more

elaborated than compiled in a related study by Schmidt & Boschuizen, 1992). Residents (63%)

relied mainly on cards of more experienced physicians.

Nendaz et al. (1998) noted that although 78% of diagnostic changes were beneficial in this

structured exercise, it was unclear whether students added diagnosis by simply copying ideas or

if they deeply understood the full meaning of their changes. Therefore extended opportunities for

discourse are recommended to ensure students do not simply mimic complied discourse of

experienced physicians. In addition authors note that the visual display of reasoning and

justifications also allows the visualization of misconceptions that could have remained

unnoticed. In this exercise, experienced physicians were provided with opportunities to examine

and refine their initial hypotheses. These finding have present strong evidence for the creation of

opportunities for physician elaborated discourse, as made possible in the End-of-Life Care

Distance Education Program in Knowledge Forum.

Identification of misconceptions and correction of errors have been and continue to be a key

concern in medical education and throughout practice (Elstein, Shulman, & Sprafka, 1978;

Friedman, Connell, Olthoff, Sinacore, & Bordage, 1998). However, verification of positive

benefits of visual display of knowledge and thinking process through collective work is a noted

shift in the recent literature related to identification of misconceptions, self-correction and related

notions of self-assessment and self-regulation.

Lester and Tritter (2001) argued for collaborative work: “A more theoretically informed

approach may be to address the genesis of medical thinking about error, through reforms of

aspects of medical education and socialization” ( p. 855). The authors regarded the failure of

individualistic reforms as due to the fact that medical errors are usually the product of a series of

factors, which suggests that change is need at a system rather than an individual level. Medical

knowledge often remains tacit, uncertainty is seldom articulated, and errors are almost never

discussed (Cimino, 1999; Patel, Arocha, & Kaufman, 1999). Lester and Tritter called for a shift


in medical culture, from a “conspiracy of silence” to one of open discourse, characterized by

collegiality and cooperation.

Most recently the effectiveness of self-directed learning has been questioned (Murad, Coto-

Yglesia, Varkey, Prokop, & Murad, 2010). Concerns around individually held misconceptions

(Lester & Tritter, 2001), tacit knowledge (Cimino, 1999), and personal reflections on practice

(Charon, 2004) pose interesting possibilities for further examination for online learning (Patel,

Yoskowitz, Arocha, & Shortliffe, 2009) and collaborative Knowledge Building (Scardamalia &

Bereiter, 2003a).

We also find articulation of issues in the medical education literature around self-assessment,

self-direction, and self-regulation (Regehr & Eva, 2006), based on early work by Kruger and

Dunning (1999) who said that individuals had difficulties in recognizing their own

incompetencies, which led to inflated self-assessments. They titled their article: “Unskilled and

Unaware of It.” In a systematic review by Davis et al. (2006), physicians were shown to have a

limited ability to accurately self-assess; in fact, a number of studies found that the worst accuracy

in self-assessment was among physicians who were the least skilled and those who were the

most confident. It is noted that self-assessment was central to continuing medical education and

lifelong learning, relicensure, specialty recertification and clinical competence for continued

practice. Hence some studies suggested that instead of continuing to try to train physicians to be

more accurate self-assessors, physicians should seek feedback from peers and health care teams,

and reflect on their clinical practice. A recent Knowledge Building summary paper identified

new ways to support self-assessment and metacogntive reflection through social construction

(Scardamalia et al., in press).

Complexity of clinical discourse will be assessed in this study using the Bordage and colleagues’

(1991, 1994, 1997) rating scale and definitions.

2.4.6 Analogies and Emergent Ideas in Abductive and Adductive Processes

It is posited that sociocognitive work in the End-of-Life Care Distance Education Program,

Knowledge Forum problem space will result in authentic discourse around emergent clinical

scenarios. Evidence of real-world, practice-based cases and communally situated explanations

and ideas articulated in the Knowledge Forum problem space may provide important examples


of the use of analogies (Glick & Holyoak, 1983; Holyoak & Thagard, 1995; Thagard, 1997) and

abductive reasoning (Thagard & Shelley, 1997), in knowledge building. Use of analogies is

advantageous for transfer of ideas to practice or what Thagard calls coherence in through and

action (Thagard, 2000, 2007).

Thagard (1997) described six uses of medical analogies that illustrate different kind of analogical

transfer: theoretical, experimental, diagnostic, therapeutic, technological, and educational

analogies. Distinction of types of analogies with the discourse is beyond the scope of this study.

However, notation of inclusion in the discourse will been seen to be indicative of abductive

reasoning.

Thagard and Shelley (1997) defined abductive reasoning as “reasoning in which explanatory

hypotheses are formed and evaluated” (p. 413). Much of knowledge building work is to create,

make sense of, and debate explanatory hypotheses or theories. Evaluation of knowledge building

discourse through examination of coherence of ideas can provide stimulus for further discourse

and opportunities for external assessment. Thagard and Shelley (1997) indicated: “Explanation is

not deduction; hypotheses are layered; abduction is sometimes creative; hypotheses may be

revolutionary; completeness is elusive; simplicity is complex; and abductive reasoning may be

visual and non-sentential” (p. 413).

Important to knowledge building discourse is Thagard’s (2007) notion of coherence. He said:

“We can judge that a scientific theory is progressively approximating the truth if it is increasing

its explanatory coherence is two key respects: broadening by explaining more phenomena, and

deepening by investigations layers of mechanisms” (p. 28).

In this study, Thagard’s (2007) concepts of broadening and deepening are used to further clarify

our understanding of knowledge building. What I call abduction in this dissertation, Thagard

calls broadening. What I call adduction is his deepening. Knowledge building work aims to

broaden and deepen ideas. Abductive and adductive knowledge work can also be considered

reflective of Bereiter’s (2000) concept of “drilling down and building up” (p. 205). These

nuances and relationships provide important clarifications and help further our understanding of

the complexities of knowledge building.


Within-note content analyses of knowledge building discourse were conducted in this study to

examine possibilities of analogical thinking, emergence, and abductive and adductive knowledge

work.

2.5 Summary and Implications

This review of the literature has demonstrated a paucity of research studies on Knowledge

Building in the context of continuing medical education. No papers were found that use social

network analysis to describe attributes and relationships amongst dimensions that successful

scaffold and support Knowledge Building.

Recent literature in continuing medical education calls for a change. The specific change

required is to improve effectiveness (Moores, Dellert, Baumann, & Rosen, 2009); it stresses the

importance of moving beyond self-learning to a community context (Dorman & Miller, 2011;

Sargeant, Mann, van der Vleuten, & Metsemakers, 2008; University of Toronto, 2011), while

remaining evidence-based (Dauphinee & Wood-Dauphinee, 2004).

New avenues and opportunities for research and development have been identified in the

literature, such the one recently published by the University of Toronto (2011): “The Office of

CEPD is seeking to change problem solving into possibilities for breakthrough thinking. This

includes thinking about continuing education as a type of creative experiment” (p. 1). It is

anticipated that recommendations such as this will enable new opportunities for creative

community engagement in continuing medical education, which will require corresponding

measures and new ways to describe social network interactions and relationships beyond those

captured by individual participation assessments.

This study seeks to fill this gap in the literature and attempts to go beyond descriptions of

individual interactions and knowledge gains to provide an understanding of Knowledge Building

community interactions, network structures, and the relationships that can successfully scaffold

knowledge improvement in continuing medical education. A new, socially constituted model of

Knowledge Building requires elaborated, detailed descriptions of relationships, as opposed to

mere statements of summative findings (Kanter, 2008). Results of this study and the analysis of

Knowledge Building in Knowledge Forum in the End-of-Life Care Distance Education Program


are intended to evaluate potentiality, to contribute to the forward progression of current

recommendations for improvement in continuing medical education.

The pedagogic design and research methods employed in this study will be described next.


CHAPTER 3

METHODS

3.1 Introduction

This chapter begins with a description of the context of this study, a statement of the goal of this

research study and the research question, followed by an overview of methodology and pedagogic

design. It then focuses on the research methods, detailing study population, instruments and

procedures, and data collection and analyses employed. This chapter concludes with a summary

table of research questions, data collection, and data analyses, framed in relation to the three study

clusters.

3.2 Context and Program Overview

This study was conducted in the context of a continuing medical education course for family

physicians, called the End-of-Life Care Distance Education Program. Participants in this

program are recruited by flyer mailed to all family medicine physicians in the Toronto, York,

and Simcoe regions, by the Office of Continuing Education and Professional Development,

Faculty of Medicine University of Toronto. Registration is open annually to the first 25

participants that respond by phone, email, or regular mail. Cost of participation is covered by the

Ontario Ministry of Health and Long-Term Care. A research ethics information and consent form

was provided to all participants (Appendix A).

The program began each year in early October and ended in early May. There were five case-

based online modules and two blended learning modules (blending videoconferenced, face-to-

face session with follow-up discussion) in the End-of-Life Care Distance Education program.

Only the five case-based online modules are included in this research study. Each case-based

module was conducted in Knowledge Forum over the course of four weeks. Each group was

facilitated by a specialist in palliative care.

The number of participants in each online group with a facilitator differed in each year and

across modules, depending on number of registrants and commitment to part or the full program.

In aggregate across 2005/2009, 73 students completed all aspects of the program (see Table 2).

Four different facilitators participated across these years.


Discourse notes written and contributed to Knowledge Forum, online pre-/posttest tests, and

attitude and opinion surveys provide the data for this research study. Data was collected across

five years, between 2004 and 2009. A mixed-methods case study methodology across years was

used.

3.3 Research Goal and Question

The goal of this thesis is to show improvements according to both traditional and non-traditional

measures—to show that there is not tradeoffs but rather improvements on both fronts. This thesis

will describe (a) traditional learning outcomes based on individual measures, (b) performance

over and above traditional measures (i.e., beyond learning as traditionally conceived and

measured), and (c) sociocognitive dynamics that enable work over and above traditional

learning.

Traditional measures of individual learning employed were online pre-/posttest outcomes and

results of attitudes and opinions surveys.

Performance measures over and above those typically concerned with traditional learning, such

as Knowledge Building measures of online activity/interactivity and social network analyses

were calculated across all groups and all years.

In-depth comparison of the sociocognitive dynamics between two groups, in the last year of this

study, 2008/2009 was also conducted, using social network analysis and content analysis. Ten

modules were analyzed to examine relationships between social network position/power.

Participants structures and ideas at the core, mid and periphery were compared with knowledge

improvement scores. Content analyses of 40% of the 2008/2009 discourse notes were performed

to gain deeper insight into facilitator/student sociocognitive dynamics, determination of

emergent themes, examination of complexity of discourse, and evaluation of evidence of

Knowledge Building indicators.

This thesis seeks to determine the following research question:


palliative care in a Web-based continuing medical education course, over a five-year

period; and if so, what social network structural relationships and sociocognitive


dynamics support knowledge improvement, and contribute to democratization and

metadesign of Knowledge Building in continuing medical education?

The study was guided by this research question, and framed by the three clusters outlined in

Table 1 and found in Chapter 2 (literature review), Chapter 3 (research methods), and Chapters

4, 5, 6, and 7 (results chapters). Details on case study methodology and pedagogic design are

discussed next, followed by detailed explanation of the research methods.

3.4 Case Study Methodology

A case study approach was used in this study (Creswell, 2003, 2007, 2009; Merriam, 2001). Both

quantitative and qualitative methods were used.

Merriam states “that the single most defining characteristic of case study research lies in the

delimiting the object of study, the case” (2001, p. 27). She further clarifies by defining case study

from other research designs using Smith’s concept of a “bounded system,” Stake’s notion of an

“integrated system” (p. 27) and Cronbach’s framing of case study research as “interpretation in

context” (Merriam, 2001).

Case study research design is particularly suited to situations where it is impossible to separate

the variables from their context, as in many holistic educational research studies.

Case studies can be further defined by three special features; they are “particularistic” meaning

they focus on a particular situation; they are “descriptive” portraying as many variables as

possible and their interactions often over time; and they are “heuristic” meaning that they are

intended to illuminate understanding of the phenomena under study (Merriam, 2001, p. 29-31).

“Previously unknown relationships and variables can be expected to emerge from case studies

leading to a rethinking of the phenomenon being studied” (Stake, 1981, p. 47, as quoted in

Merriam, 2001, p. 30).

Case study research is not limited by any particular methods for data collection or data analysis.

Nor is it limited by any predefined frameworks, such as those used in program evaluation

(Posavac & Carey, 2003). Mixed-methods research focuses on collecting and analyzing both

quantitative and qualitative data in a single study to provide deeper understanding and

convergence of findings from different sources of data (Creswell, 2003).


In the current study I employ various quantitative methods, including those typically used to

evaluate individual learning and those more recently developed to assess collective Knowledge

Building (Burtis, 2001; Teplovs, 2010; Teplovs & Scardamalia, 2007) and social network

relationships. Traditional quantitative methods are used to evaluate knowledge gains, through

pre-/posttests, and participant attitudes and opinions surveys (Dillman, 1978, 2000). The

Knowledge Forum Analytic Toolkit, is a powerful suit of analyses tools within Knowledge

Forum that provide numerical analyses and graphical representations regarding online

contribution and participation patterns (contributing notes, building on, annotating, and

referencing notes of others, etc.). Social network analyses, described in detail in the current

literature review, is used to describe relationships of participant structures, idea interactions, and

sociocognitive dynamics (Scott, 2001; Scott & Carrington, 2011; Scott, Carrington &

Wasserman, 2005).

I employ qualitative analysis on a small subset of data in this study, as Creswell (2007) indicates,

to enrich our understanding and verify relationships. I use within-note, content analyses of the

discourse in Knowledge Forum. Inductive and deductive approaches are employed (Creswell,

2007; Merriam, 2001). Discourse is analyzed deductively according to a framework of

teacher/student structural/power relationships (Mills, 1997; Hodges, Kuper, & Reeves, 2008) and

definitions of clinical complexity (Bordage, 1994), and definitions of Knowledge Building

(Scardamalia, 2002), and inductively for emergent themes. Chi’s method (1997) of quantifying

qualitative analyses is applied to the deductive analyses.

This mixed-methods multiyear case study was conducted in the unique context of the End-of-

Life Care Distance Education Program, with a study population of physicians/students and

facilitators/palliative care experts, using a series of quantitative and qualitative methods, to

describe important relationships between individual learning and collective Knowledge Building.

The research protocol for this study was approved by the Health Sciences Research Ethics Board

of the University of Toronto in 2004.

3.5 Pedagogic Design

The End-of-Life Care Distance Education Program was specifically created to fill a gap in

palliative care education for continuing medical education for physicians in practice. The


pedagogic design of the program was based on Knowledge Building theory and the use of

Knowledge Forum for collective discourse. However, the curriculum remained rooted in a

traditional mode, framed by predefined learning objectives and a clinical case scenario.

Participants were divided into groups facilitated by an expert in palliative care. Framework for

facilitation was extended beyond that described in problem-based learning (Barrows and

Tamblyn, 1980), as a guide-on-the-side. In this course facilitators were framed as full

participants (Bereiter & Scardamalia, 1993).

3.5.1 Multimedia Design of Knowledge Forum Communal Space

A goal underlying the design of Knowledge Building environments (Scardamalia, 2003b) is to

create communities for knowledge advancement, enabled by supports for high level processes

and systems for embedded and transformative assessment within the discourse. Knowledge

Forum, is specifically design to support Knowledge Building and was used in the current study.

A website (End-of-Life Care Distance Education, 2004) was created to provide a publically

accessible information and access to the End-of-Life Care Distance Education Program. The

website contains information on the course, registration, the schedule, credits, and access to

online resources and Knowledge Forum (Appendix B).

The program opens with a videoconferenced face-to-face session, called Introduction to

Palliative Care. The next three modules are online cases. Mr. Singh’s Pain, Parts 1 and 2, focuses

on management of neuropathic pain and the case of Mary’s Misery deals with complex pain and

psychosocial issues. Next is an interactive videoconferenced, face-to-face session, exploring

palliative care symptoms other than pain (e.g., dyspnea). This session is followed by the last two

online cases, called Judy’s Last Days, Parts 1 and 2, which deal with patient and family issues

around death and dying.

Each multimedia case scenario in Knowledge Forum begins with a text-based introduction,

followed by a 3 to 5–minute clinical video vignette of the patient, followed by a prompt for

collaborative knowledge building issues for discussion (Appendix C).

Individual and collective “Reflection-on-Practice” components are integrated within each

scenario to draw out case relevance to real-world practice for emergent discussion. Digital


references were organized by case and clinical topic, in an online library to support evidence-

based knowledge building. “Ideas as the Centre” (Scardamalia, 2003a) and related to the case

objectives, are provided to guide the collective discourse. Predetermined learning objectives are

posted for each online module. Each of the five online modules were conducted over a one-month

period of time in Knowledge Forum.

3.5.2 Knowledge Building Theory-Based Design

Knowledge Building theory-based design was the pedagogic approach employed throughout all

modules (Scardamalia and Bereiter, 2003a, 2003b). The End-of-Life Care Distance Education

Program was designed to provide a (media communal space for collaborative Knowledge

Building, with an emphasis on the Knowledge Building principles of community responsibility,

improvable ideas, embedded and transformation assessment through discourse, and

democratization of ideas (Scardamalia, 2002). The goal in the creation of a Knowledge Building

community is to engage learners in a complex, interactive process that enables them to take

charge of the educational process at the highest levels. As in the case of knowledge-creating

organizations, participants see ideas as improvable, and their goal as improving them (Nonaka,

1991). Scardamalia and Bereiter (2005) argue that complexity theory or a systems approach to

learning arises from the need to address two large problems--creativity and depth of

understanding.

Pedagogic design of the End-of-Life Care Distance Education Program can be said to reflect the

intersection of prescribed learning/assessment, as demonstrated by the predefined learning

objectives and opportunities for emergent knowledge building discourse and assessment.

Learning objectives from the three pain modules were also used to categorize subject matter

knowledge on analysis of pain pre-/posttests. The pain pretest is administered before the first

pain module, Mr. Singh’s Pain, Part 1 and the pain posttest is administered after the third pain

module, Mary’s Misery. Three other sets of pre-/posttests were administered to participants but

were excluded in this study since they were qualitative in nature.

All online pre-/posttest were developed primarily for purpose of scaffolding self-assessment, to

provide constructive feedback for individual improvement goals and collective Knowledge

Building objectives. The difficulty of accurately assessing one’s own strengths and weaknesses


is acknowledged in the research literature (Donovan et al., 1999; Eva & Regehr, 2005; Regehr,

Hodges, Tiberius, et al., 1996). Pre- and posttests are commonly used in a summative rather than

a formative manner, to make claims about change in educational outcomes. In this study,

individual formative feedback was provided to participants online to support self-assessment,

enabling participants to identify knowledge “lacks” (Bereiter, 2002a; Scardamalia & Bereiter,

1989, 2005).

A knowledge self-assessment system, such as this, is intended to enable individual conceptual

change, as well as elevate the community Knowledge Building discourse; the ultimate goal is to

evoke deep relational understanding (Bereiter, 2002c), and to translate higher levels of

knowledge into new ideas, improved practice, and ultimately more effective patient care. Further

information on the novel pre-/posttest pedagogic design created for this program can be found

elsewhere (Lax, Singh, Librach, & Scardamalia, 2006).

The Knowledge Forum discourse environment opens opportunities for work in belief and design

modes (Bereiter & Scardamalia, 2003). The design of the discourse environment can be

characterized as open and unstructured, yet scaffolded to support Knowledge Building and

learning as a by-product of Knowledge Building. Traditionally instructional scaffolds have been

used in learning environments as temporary supports for learning that are gradually removed as

the learner achieves the intended outcome (Brown, Collins, & Duguid, 1989; Brown & Palincsar,

1989; Collins et al., 1989; Vygotsky, 1978). In a Knowledge Building environment, scaffolds are

permanent features of the environment since there is no predetermined level of achievement and

the aim is to support continual knowledge improvement (Lee, Chan, & Van Alst, 2005;

Scardamalia & Bereiter, 2003b). In the End-of-Life Care Distance Education Program,

Knowledge Building scaffolds are represented in Knowledge Forum both technically and

conceptually, within the discourse, through community responsibility, including participation of

content specialists, as well as, through learning objects and intentional efforts to raise knowledge

work beyond the predefined objectives. Authoritative resources in the online library also provide

information and evidence to scaffold Knowledge Building (Scardamalia, 2002a).

Unstructured online discourse environments have raised some concern in the literature about the

perpetuation of misconceptions (Burtis, Chan, Hewitt, Scardamalia, & Bereiter, 1993) and

possibility of misconceptions not being addressed (Hewitt, 2005). The role of the content expert


is primarily to challenge uncaught misconceptions and if need be didactically correct them. As

well as, in the End-of-Life Care program facilitators can provide authentic, practice-based

knowledge on issues such as billing, hospital-based versus community-based practice, etc.

3.6 Research Methods

The research protocol was reviewed and approved by the University of Toronto ethics board.

This study was conducted from 2004 to 2009. Pedagogic design, including media and cases,

were developed in Knowledge Forum 2003/2004. A pilot study was conducted from October

2004 to May 2005. Feedback on the pilot study indicated the need for change to the pain

modules and pre- and posttests, to include a medication titration chart. Otherwise content

remained the same throughout this study from 2005 to 2009. Interactivity design of the

Knowledge Forum module notes were changed in 2008/2009. The lengthy scrolling module note

was changed to an interactive series of pages, like a mini-website, to improve organization.

This study is concerned with evaluation of the End-of-Life Care Distance Education Program

across five years, the 2004/2005 pilot and following four years: 2005/2006, 2006/2007,

2007/2008, 2008/2009. Across all five years, five modules in Knowledge Forum were examined:

the three pain content modules, Mr. Singh’s Pain, Part 1, Mr. Singh’s Pain Part 2, and Mary’s

Misery and the two last days modules, Judy’s Last Days, Part 1 and Judy’s Last Days, Part 2.

Online discourse in Knowledge Forum for each module was conducted over a one-month time

period.

A mixed-methods case study approach was used (Creswell, 2003, 2009; Merriam, 2001).

Measures of knowledge change, in combination with contribution and interaction measures,

social network measures, content analyses and Knowledge Building indicators, are used to

address important outcomes across years, in the absence of control-group contrasts.

Although a design research approach (Bereiter, 2002b; Brown, 1992; Collins, Joseph, &

Bielaczyc, 2004) is generally well matched to Knowledge Building research, it was not the case

in this study. Few design changes, other than in the first year of the study, were made, so design

research was not employed.


3.6.1 Participants

Thirteen family physicians complete all aspects of the first program, pilot study. In the following

four years, 73 family physicians completed all aspects of the program and research study. In

addition four facilitators participated across eight groups over five years (Table 2).

Family physicians practicing in three different regions of southern Ontario: Simcoe, Toronto and

York, were invited by email and mailed brochure to participant in the study and program. One or

two specialists in palliative care medicine were chosen to facilitate discourse each year, based on

the number of participants registered in the program. All facilitators were in current practice as

clinicians in a large Toronto teaching hospital and were hand-selected by the content developer

of the program, Dr. Anita Singh. All facilitators were considered equally knowledgeable.

Participants were divided into groups for online collaborative work in Knowledge Forum.

Participation across the five years was as follows:

· In 2004/2005 pilot study, 18 family physicians and two facilitators participated in the

course (two groups); 16 family physicians completed the pain modules, and 13

completed the course.

· In 2005/2006, 16 family physicians and one facilitator participated in the course (one

group); 15 family physicians completed the pain modules, and 14 completed the

course

· In 2006-2007, 14 family physicians and one facilitator participated (one group); 11

family physicians completed the pain modules, and the full course.

· In 2007/2008, 31 family physicians and two facilitators participated (two groups); 26

family physicians completed the pain modules, and the full course.

· In 2008/2009, 22 family physicians and two facilitators participated; 18 family

physicians completed the pain modules, and 15 completed the course.

3.6.2 Instrumentation and Procedures

Pain pre- and posttests of knowledge, Knowledge Forum Analytic Toolkit and new tools for

measures of online discourse activity and interactivity in five modules were used, along with a

postcourse, self-reported, attitudes and opinions survey for data collection and analyses in this

study (Table 2).


3.6.2.1 Pain knowledge pre- and posttests. Pain pre- and posttests were developed for online

use. All tests were individually password protected (but can be accessed online with

username/password=test/test). Learning objectives were categorized for the acute and complex

pain modules, and guided pre-/posttest design, development and feedback. Twenty-eight

multiple choice, fill-in-the-blank, and short answer/word recognition questions were constructed

based on the following four learning objectives:

· Understanding principles of pain management = six questions

· Use of opioids in cancer pain = seven questions

· Understanding neuropathic pain = five questions

· Understanding bone pain = 10 questions

Questions on the pain pre-/posttests were exactly the same. Tests were completed online

and automatically dated and time stamped. Upon completion of the pretest participants were

given individual Score Cards based on the four pain objectives to aid self-assessment and

identification areas of strength and weakness, particularly knowledge “lacks” and/or possible

misconceptions. Participants were asked to read references provided in the online library

(according to objectives) and collaboratively contribute to the online discourse, focusing on

necessary areas for Knowledge Building.

Upon completion of the three pain modules, after three months of online discourse, participants

were asked to complete the pain knowledge posttest. Data was collected online. Participants

were given individual comparative pre-/posttest Score Cards, the correct answers, in depth

explanations for each answer, and references to individually address any outstanding knowledge

issues around the subject of pain in palliative care.


Table 2

End-of-Life Care Distance Education Program Participants and Data Collection Instruments

EVALUATION COMPONENTS AND NUMBER OF PARTICIPANTS

2004/2005

(Pilot)

2005/2006 2006/2007 2007/2008 2008/2009 2005–2009

Students Students Students Students Students

G1 G2 G1 G1 G1 G2 G1 G2 Total

Pain Pretest

(matched)

9 11 7 26 10 7 61/73 =

83.56%

Mr. Singh’s

Pain, P1

8 11 15 11 14 17 10 9

Mr. Singh’s

Pain, P2

8 11 15 11 15 16 10 9

Mary’s

Misery

16 15 11 14 16 10 9

Pain Posttest

(matched)

9 11 7 26 10 7 61/73 =

83.56%

Last Days P1 14 15 11 13 16 10 9

Last Days P2 13 15 11 13 15 10 9

Attitudes and

Opinions

Survey

12/13

(77%)

12/15

(80%)

10/11

(91%)

26/28

(93%)

12/19

(63%)

60/73 =

82%

Facilitators 1A 1B 1C 1C 1C 1D 1C 1D

Note. Facilitator A (palliative care physician—Simcoe) facilitated one group in the Pilot Study. Facilitator B

(palliative care physician—Sinai) facilitated one group in the Pilot Study. Facilitator C (palliative care physician-

Sunnybrook) facilitated one group for four years. Facilitator D (palliative care physician—Sunnybrook) facilitated

one group for two years (took the course first).

3.6.2.2 Attitudes and opinions survey. An online Attitudes and Opinions Survey was

constructed, using a 5-point Likert scale and Yes/No questions to examine participants’

satisfaction with their continuing education experience and collaborative knowledge building

(Dillman, 1978, 2000). The survey was linked in a Knowledge Forum note, and data was

collected online. Participants received an email invitation to complete it, at the conclusion of the

course, and a reminder email two weeks later.


3.6.2.3 Knowledge Forum online activity and interactivity. Five (media, case-based modules

were created in Knowledge Forum for collaborative knowledge building around palliative and

end-of-life care issues. Students and facilitators worked on each module for a one-month period,

writing, building-on, and reading each other’s notes in Knowledge Forum. These notes represent

data collected in Knowledge Forum. Learning objectives were provided online to participants for

each module. These objectives were designed by course content developers, experts in palliative

care and deemed to represent the body of knowledge to be acquired in this course. Table 2

provides a summary of participants and data collection instruments used in this study.

3.6.3 Data Collection and Analyses: Pilot Study 2004/2005

Participants completed pain pre- and posttests online. Data was collected automatically,

downloaded to Excel, matched, and then analyzed in SPSS. Pre to posttest scores were analyzed

for significant difference and effect size. Attitudes and opinions on the program were collected

by online survey. Data was time and date stamped. Analysis was performed in SPSS. Online

activity and interactivity were calculated using the Knowledge Forum Analytic Toolkit for read,

write, and build-on measures.

3.6.4 Data Collection and Analyses Cluster 1: Traditional Measures Across Years

2005/2009

3.6.4.1 Analyses of pain knowledge improvement. Participants’ pre- and posttest data were

automatically collected online (date and time stamped). These data were downloaded, matched

and scored in Excel and SPSS. Some participants completed only the pretest; these unmatched

data were scored but not used. The 2004/2005 pilot study data was not included in the

cumulative analysis since changes were made after the 2004/2005 course to specific pre- and

posttest questions to validate tests for continued use. Pain Knowledge pre- and posttests matched

results were analyzed by year, for each of the four years and cumulatively, across 2005–2009.

3.6.4.2 Attitudes and opinions of collaborative online work. Online data from the Attitudes

and Opinions Survey were collected automatically, date and time stamped, and downloaded to

Excel files for descriptive analysis in SPSS. Data was analyzed by year and cumulatively.

Cumulative analysis of the 2005–2009 results are reported in the current study.


3.6.5 Data Collection and Analyses Cluster 2: Activity, Interactivity, and Social Network

Measures Over and Above Traditional Measures

Online activity statistics were generated by the Analytic Toolkit (ATK) in Knowledge Forum.

This is a powerful set of tools that can analyze data in numerous ways. The ATK was used in this

study to analyze read/write statistics, measures of interactivity, and more complex correlations

involving social network analyses.

3.6.5.1 Knowledge Forum online activity and interactivity. All asynchronous online notes

written and contributed by participants are saved on the server and appear within Knowledge

Forum. The Knowledge Forum Analytic Toolkit was used to calculate activity measures, i.e.

number of notes created and percent of notes read, as well as, interactivity measures (i.e., percent

of notes built-ons). These measures were calculated across all groups and at the conclusion of

each of the five online modules. Participant, including facilitator, measures were performed for

program years 2005/2006, 2006/2007, 2007/2008 and 2008/2009.

3.6.5.2 Knowledge Forum graphic contribution and social network analyses. In 2008, the

Knowledge Forum ATK was replaced with a more extensive set of analytic tools, including

graphic contribution tools and social network analysis tools. These new Knowledge Forum

assessment tools were used to analyze two groups of data in the 2008/2009 End-of-Life Care

Distance Education Program. Participant read/write contributions of 2008/2009 Group 1 and

Group 2 were analyzed along with social network measures of network links and network

density of who built-on whose notes and who read whose notes.

3.6.5.3 2-way ANOVA. A 2-way ANOVA was performed, in SPSS, on pain knowledge pre-

/posttest results to determine if there was a significant difference across and between 2008/2009

Group 1 and Group 2.

3.6.5.4 Social network structural analyses. Detailed social network analyses were conducted

on the 2008/2009 Knowledge Forum Analytic Toolkit results of participant notes built-on.

Netminer 3© software was used for structural analyses on 2008/2009 Group 1 and 2 discourse

across all 10 modules. The following social network structural analyses were conducted:

· Frequency, network edges, network density, Eigenvector centrality, in-degree centrality,

out-degree centrality


· Cohesion index, cliques

Social network analyses of 2008/2009 Groups 1 and 2 examined structural dimensions over time

and between networks (by student only and by student/facilitator) to compare and contrast

dimensions, to determine what social network relationships support knowledge improvement

and democratization of participation.

3.6.5.5 Significant difference and effect size of social network density and centrality

measures between groups, with and without facilitator. Social network measures between

2008/2009 Group 1 and Group 2 were analyzed for significant difference (t tests) and effect size

(Cohen’s d) in SPSS. These analyses were conducted with and without the facilitator for each

group, across all modules, to determine over facilitator effect.

A second analysis was conducted to determine significant difference and effect size between

Group 1 and Group 2, with facilitator, in regards to network density and centrality measures of

build-on notes and notes read. Network centrality measures in both analyses included

Eigenvector centrality, in-degree, and out-degree.

3.6.5.6 Relationship of social network structural analyses of the three pain modules to

pain pre-/posttest scores. Clique and cohesion measures were compared between 2008/2009

Group 1 and Group 2 for the three pain modules; pain pre-/posttest results were compared.

Next, tests for significant difference and effect size were conducted on measures of density and

centrality using the three pain modules. Because of the low numbers nonparametric correlation

(Spearman’s rho) was used. Groups 1 and 2 were combined for Spearman’s correlation of pain

posttest score with social network centrally variables.

3.6.6. Data Collection and Analyses Cluster 3: Social Network Analyses and Measures of

Sociocognitive Dynamics That Support Knowledge Building Outcomes, Over and Above

Learning

3.6.6.1 Social network position and power analyses. This social network analysis was

conducted based on the ATK data from the 2008/2009 Group 1 and 2 discourse across all 10


modules. Netminer 3 software was used for centrality position and idea power analyses of build-

on notes.1

3.6.6.2 Social network content analyses.

3.6.6.2.1 Social network analysis of facilitator and student patterns of discourse. Forty

percent of the 2008/2009 dataset were analyzed. The same two modules in Group 1 and Group 2

were selected: Mr. Singh’s Pain, Part 2 and Judy’s Last Days, Part 2. Selection was made by the

researcher based on highest core performance in previous analyses of shared power.

Two analyses were conducted. The first examined patterns of facilitator discourse. Facilitator

participant structures were categorized according to a modified protocol based on Tabak and

Baumgartner (2004, Table 1, p. 403), definitions of “monitor, mentor, or partner” categories. I

added the category of expert to highlight episodes of didactic teaching. I also added

subcategories of partner/expert and mentor/partner since some facilitator notes exhibited both

stances. This analysis was scored by a second rater for interrater reliability of findings.

The second analysis in this section examined patterns of discourse statements and questions.

Unlike Zhang et al.’s (2002) study, based on advancement of Knowledge Building through

statements, this study takes the perspective that Knowledge Building is advance not only by

statements but also by questions. In this study, I ask: Who asks the questions students or the

facilitator and from what social network position? These analyses can be considered a form of

qualitative social network analysis of facilitator/student participant structures.

3.6.6.2.2 Content analysis of themes of discourse, beyond learning objectives. Forty

percent of the 2008/2009 dataset were analyzed. The same two modules in Group 1 and Group 2

were selected: Mr. Singh’s Pain, Part 2 and Judy’s Last Days, Part 2. Within-note qualitative

analysis was conducted; threads of discourse were categorized (Zhang, et.al, 2002). Themes were

identified as emergent, beyond those listed in the predetermined learning objectives, or related to

the learning objectives. Emergent ideas were reviewed as abductive and adductive knowledge

work—that is, whether they broadened or deepened ideas.

1 Note. Social network measures of notes read was consistently high across all modules and similar between

groups and therefore no further analyses were done on these; only build-on notes were used in further analyses since

they were more variable and offered the possibility to demonstrate difference and gain a deeper understanding.


3.6.6.2.3 Complexity of clinical discourse. Group 1 and Group 2 notes in the 2008/2009

dataset were analyzed. The same two modules in each Group were scored. Within-note discourse

was analyzed using the Semantic Analysis of Clinical Discourse scale created by Dr. George

Bordage (1994). Knowledge Forum notes were scored on a 4-point scale, rating discourse as

reduced, dispersed, elaborated, or compiled. Administrative notes were placed in a separate

category. This analysis was scored by a second rater for interrater reliability of findings.

3.6.6.2.4 Evidence of Knowledge Building indicators and exemplars. Analyses of

2008/2009 Groups 1 and 2 Knowledge Forum notes were conducted. Four modules (40%) from

the 2008/2009 dataset were analyzed; the same two modules from Group 1 and the same two

modules from Group 2. Evidence of Knowledge Building indicators was scored using Sibbald’s

(2009) scoring matrix based on Scardamalia’s (2002) definitions of the 12 principles of

Knowledge Building. This analysis was scored by a second rater. Exemplars of Knowledge

Building indicators were selected from the dataset. Knowledge Building indicators are used to

establish evidence of Knowledge Building sociocognitive dynamics, such as community

responsibility, idea improvement, democratization of knowledge, epistemic agency and emergent

work demonstrating cocreation of knowledge and metadesign of curriculum in a continuing

medical education course.

3.6.7 Summary

Various data collection and data analyses methods were used in this mix methods, multiyear,

case study. Quantitative statistical analyses, social network analyses, and qualitative analyses

were conducted. Table 3 summarizes demographics of participants, and Table 4 summarizes

research questions and the associated data sources and analyses.


Table 3

Demographics of the Study Population

DEMOGRAPHICS 2005/2009

Respondents (N=60/73) 82%

1. Gender Male Female n/a

35(58.3%) 24(40%) 1(1.7%)

2. Region of practice Toronto York Simcoe/other

29(43.8%) 13(21.7%) 15(26.7%)

3. Area of practice/specialization Family medicine Specialist n/a

56(93.3%) 3(5%) 1(1.7%)

4. Year of graduation 1958-2008

5. Years in practice 0-5 5-10 11-20 More than 20 years

9(15%) 9(15%) 15(25%) 25(41.7%)

Table 4

Research Question/Subquestions, Data Analysis, and Data Sources

RESEARCH QUESTION

Does Knowledge Building improve physicians’ knowledge and understanding of palliative care in a Web-based

continuing medical education course, over a five-year period; and if so, what social network structural

relationships and sociocognitive dynamics support knowledge improvement, and contribute to democratization

and metadesign of Knowledge Building in continuing medical education?

SUBQUESTIONS

CLUSTER 1 TRADITIONAL OUTCOME MEASURES

Subquestion 1 Did students’ pain knowledge improve from pre to posttest and if so, is this improvement

significant and what is the effect size?

Data Analysis Data Sources

2-tailed t-test Matched pre-/posttests (Pilot and four years)

Cohen’s d effect size Matched pre-/posttests (Pilot and four years)

Subquestion 2 Traditionally continuing medical education is a self-directed or didactic experience; what are

participants’ attitudes and opinions towards collaborative online knowledge building in the

End-of-Life Care Program?


Likert scale and Yes/No questions Online survey (Pilot and four years)


Table 4 (continued)

CLUSTER 2 PERFORMANCE OVER AND ABOVE TRADITIONAL MEASURES

Subquestion 3 What are participants’ online activity and interactivity measures, by module, by group, by

year, in the 2004/2005 pilot and 2005–2009 study (in each of the five modules, across two

groups, and over four-years)?


ATK tools in Knowledge Forum Five online modules, by group each year (Pilot

and four years)

SNA tools in Knowledge Forum Ten online modules, over two groups

(2008/2009)

Subquestion 4 Is there a significant difference in 2008/2009 Groups 1 and 2 pain knowledge scores from

pretest to posttest, and is there a significant difference between groups?


2-way ANOVA 2008/2009 Groups 1 and 2 pre-/posttests

Subquestion 5 Pain knowledge improved significantly in both groups; however, knowledge increase was

greater in Group 1 than in Group 2. What are the social network structural differences

between groups that supported increased knowledge improvement and how are these

differences related to centralization and/or democratization of participation and ideas?

Structural SNA Data Analysis Data Sources

Network edges

(measures and visualizations)

2008/2009 Groups 1 and 2, five modules,

build-ons and notes read, with/without

facilitator = 20 in total

Network density





Eigenvector centrality





In-degree centrality





Out-degree centrality





Cohesion index and cliques

(measures: cohesion of ideas, no. of

cliques, no. of members, members, etc.)


build-on notes, with facilitator

Correlation of social network centrality

measures and knowledge gains

(Spearman’s rho)

2008/2009 Groups 1 and 2, SNA of three pain

modules and pain posttest outcomes


Table 4 (continued)

CLUSTER 3 SOCIOCOGNITIVE DYNAMICS THAT ENABLE WORK OVER AND ABOVE

TRADITIONAL LEARNING

Subquestion

6

What are the social network relationships between structural position, power (defined as

centrality of ideas), and knowledge improvement; and how are these relationships reflected in

facilitator/student sociocognitive dynamics of Knowledge Building (beyond learning objectives),

through emergent themes, in complexity of discourse, and by indicators of Knowledge Building?

Data Analyses Data Sources

a. NETWORK POSITION AND POWER

(measures and mapped visualizations)

Structural analysis of position/idea centrality/power

and relationship to knowledge improvement


build-ons (with facilitator),

= 10 social network centrality maps

Identification of whose ideas are at the core, mid,

or periphery (facilitator and/or students; Aviv

et al., 2003)

Relationship of student network position to

individual pre-/posttest knowledge gain Pre-/posttest results by individual and

social network centrality maps

b. FACILITATOR/STUDENT SOCIOCOGNITIVE DYNAMICS

Facilitator stance

Facilitator as monitor, mentor, participant, or

expert (modified protocol by Tabak &

Baumgartner, 2004)

2008/2009 Groups 1 and 2, four modules

(40% of dataset) to be scored by PI (and

20% by second rater), within-note analysis

Who asks the questions that drive knowledge

work?

As above

c. ANALYSIS OF EMERGENT THEMES (OVER AND ABOVE

LEARNING OBJECTIVES)

Categorization of themes and threads based on

predefined learning objectives and emergent ideas

(Zhang et al., 2009)


(40% of dataset) to be scored by PI,

within-note analysis

Relationship of emergent themes to abductive and

adductive knowledge improvement

As above

d. ANALYSIS OF COMPLEXITY OF DISCOURSE

Use of semantic analysis of clinical discourse scale

(Bordage, 1994), 4-pt rating scale of clinical

discourse as reduced, dispersed, elaborated, or

compiled



30% by second rater)

e. ANALYSIS OF KNOWLEDGE BUILDING INDICATORS

Evidence of Knowledge Building principles,

particularly community, democracy, agency, and

improvable ideas related to Knowledge Building

intentionality and emergent ideas/themes


(40% of dataset) to be scored by PI and

second rater

Use of scoring protocol by Sibbald (2007) and

definitions by Scardamalia (2002)

78

CHAPTER 4

RESULTS—PILOT STUDY

4.1 Introduction

The End-of-Life Care (EoL Care) Distance Education program pilot study was conducted

between September 2004 and May 2005. Traditional pre-/posttest and attitude and opinion

survey measures were employed, as well as online activity and interactivity measures (that go

beyond these traditional measures) from five online discourse modules in Knowledge Forum.

Typical online discourse read/write measures were collected along with the more innovative,

collective Knowledge Building measure of build-on notes. A total of 19 students and two

facilitators participated in the first module; Group 1 was composed of eight family physicians

plus an expert in palliative care, and Group 2 was composed of 11 family physicians and an

expert in palliative care. Groups were amalgamated in the third module and 13 students

completed the course. The pilot study was guided by the following research question:


palliative care in a Web-based continuing medical education course, and if so, what

social network structural relationships and sociocognitive dynamics support knowledge

improvement and contribute to democratization and metadesign of Knowledge Building

in continuing medical education?

Pilot study results were used as formative feedback for iterative improvements in design and

instrument validation.

4.2 Pilot Study, 2004/2005: Results

4.2.1 Pain Knowledge, 2004/2005: Pre- and Posttest Results

Nine participants completed both the pain pre- and posttests (for the pain online modules, Mr.

Singh, Part 1; Mr. Singh, Part 2; and Mary’s Misery). Pre- and posttests were analyzed by each

of the four pain objectives as well total score (Table 5). Mean scores, standard deviations,

confidence intervals, and effect sizes were calculated in regards to the 2004/2005 pilot test study.

A paired t-test was used to determine statistical change, across four objectives and in total. The

objectives were:

79

1. Understanding principles of pain management

2. Use of opioids in cancer pain

3. Understanding neuropathic pain

4. Understanding bone pain

Table 5

Matched Results From the EoL Care Pain Pre-/Posttests, 2004/2005

2004/2005 (N = 9)

Objective Pretest Posttest

M (SD) 95% CI M (SD) 95% CI t(8) p d

1. Pain Management 0.91 (0.17) [0.79, 1.02] 0.98 (0.06) [0.94, 1.02] 1.84 0.104 0.59

2. Opioid Use 0.40 (0.15) [0.29, 0.50] 0.47 (0.14) [0.37, 0.56] 1.28 0.237 0.49

3. Neuropathic Pain 0.68 (0.26) [0.51, 0.86] 0.83 (0.19) [0.70, 0.97] 1.61 0.147 0.66

4. Bone Pain 0.60 (0.18) [0.47, 0.72] 0.81 (0.08) [0.76, 0.87] 3.15 0.014 1.53

Total 0.54 (0.14) [0.44, 0.64] 0.65 (0.10) [0.58, 0.72] 2.35 0.047 0.89

Objective 1, Pain Management, demonstrated a very high score on the pretest and a higher score

on the posttest. Although the change was not significant, this was likely due to the ceiling effect

on pretest. Mean pretest score was 91% and posttest score was 98%. Thus, there was not much

room for improvement based on what seems to be a high level of understanding or too easy

questions in this section.

Objective 2 results, the use of opioids, also failed to show statistically significant increases in

scores. However, the reason appears very different than that seen in Objective 1. For Objective 2,

pretest scores were very low, 40%, as were confidence intervals; posttest scores were only

moderately higher, 47%. Questions and answers related to Objective 2, Opioid Use, were

carefully examined. It was found that students needed a standardized chart to optimize opioid

titration. This chart was added to the online pre- and posttests as well as embedded in the online

environment for use the following year. In addition facilitators were told in the current year and

following years that this area would require attention and much focus in the discourse.

Objective 3, Neuropathic Pain, presented positive and significant change; the mean score rose

14%, from 60% on the pretest to 83% on the posttest. Similarly, results from Objective 4 on bone

pain demonstrated a significant increase in mean scores from 54% on pretest to 81% on posttest.

80

Overall the total mean score across all objectives demonstrated significant improvement of 11%

on paired t-test = 2.35, p <.05, from pretest (54%) to posttest (65%).

Cohen’s d statistic was used to measure effect size. Guidelines used to interpret the statistic

were: .20 small; .50 medium; and .80 or more, indicative of a large effect. Effect size of the

2004/2005 program was 0.89.

To establish validity of pain pre-/posttests all questions were analyzed by item to identify

problematic questions. Items were scored and McNemar tests computed to assess whether the

difference was significant. Because of the small N (9), the power of the tests was not high.

However, these results were useful to identify problematic questions, where scores were low or

where there was a decrease or negative outcome from pre- to posttest. Graphs based on this

analysis clearly demonstrate items with positive change from pre- to posttest and items with zero

or negative change from pre- to posttest (Appendix D). Problematic questions (i.e., questions 2,

18 and 11) and answers were reviewed. Wording was changed. Titration (part questions 8, 10,

and 11, were reviewed and the need for inclusion of a titration chart in future years was

identified. Item analysis would be run again in 2005/2006 for comparison purposes and to ensure

adequate changes had been made to address problematic questions identified on the 2004/2005

pre- and posttests (Appendices D and E).

4.2.2 Pilot Study, 2004/2005: Attitudes and Opinions Survey Results

Seventy-seven percent (10 out of 13) of the participants responded to the Attitudes and Opinions

Survey (Table 6). Only one respondent had taken an e-learning course previously. One hundred

percent of respondents indicated that this continuing education experience was useful and that

they acquired new knowledge. Eighty percent of respondents indicated that as a result of this

course they would change their approach to palliative care.

Notable is the fact that pre-/posttest embedded feedback was considered helpful by 100% of

respondents. In addition, 90% indicated that this type of embedded feedback helped them

determine their individual learning needs. One hundred percent of respondents indicated

collaborative knowledge building helped them self-assess their strengths, weaknesses and

knowledge gaps, and reflect on different perspectives.

81

Most respondents (80%) indicated that the digital resources were easy to access in the online

library, called My.library. However, 40% indicated that ease of access did not prompt citation of

evidence in online discourse.

Table 6

Responses to Attitudes and Opinions Survey, 2004/2005

Demographics (n = 10 of 13)

1. My practice is located in the region of 2. I have been in practice

Simcoe 3 0–5 years 3

Toronto 6 6–20 years 3

York 1 Over 20 years 4

Continuing professional development feedback Yes (n) No (n)

3. Did you find this continuing education experience useful? 10 0

4. Did you acquire new knowledge? 10 0

5. Do you think as a result of this course you will change your approach to

palliative care? 8 2

e-Learning

6. I have taken a course using e-learning (i.e. online collaborative learning)

courseware before this one. 1 9

7. Digital resources in My.library were easy to access. 8 2

8. Access to digital references prompted use of evidence-based resources in

online discussion. 6 4

9. Pre- and posttest feedback was helpful. 10 0

10. Pre- and posttest feedback helped me determine my learning needs. 9 1

Collaborative Knowledge Building Outcomes

11. Collaborative knowledge building discourse in KF helped me reflect on

different perspective presented. 10 0

12. Awareness of my own knowledge, attitudes and opinions helped me to

self-assess my strengths, weaknesses and/or knowledge gaps. 10 0

13. Collaborative knowledge building helped me to identify changes I

would like to make in my practice. 9 1

14. Individual and collaborative reflection-on-practice sections were

helpful. 10 0

Overall evaluation Equivocal Agree Strongly agree

15. I would like to see more distance education programs for

Continuing Medical Education made available. 2 2 6

16. Based on my experience of collaborative e-learning, I

would recommend it to a colleague. 1 6 3

Average

Above

average Excellent

17. Overall I would rate the online collaborative knowledge

building component of this program as: 2 6 2

18. Overall I would rate the videoconferencing component of

this program as: 1 5 4

82

Eighty percent of respondents would like to see more continuing medical education courses

online and 90% indicated they would recommend this particular course to a friend. Overall, 10 of

13 (80%) of respondents rated the Knowledge Building component of this program, on a 5-point

Likert scale, as above average or excellent. Ninety percent rated the videoconferencing

component as above average or excellent. Use of embedded pre-/posttests as formative feedback

was determined to be helpful to participants in identifying learning needs and knowledge gaps.

4.2.3 Pilot Study, 2004/2005: Online Activity Analytic Toolkit Measures

Knowledge Forum Analytic Toolkit results suggest strong levels of online read/write activity

(Table 7). Of particular significance is the percentage of notes read (74% to 85%) by participants

across all modules. Results of percent of notes linked in Group 1 does not reach over 50% in the

first two modules; in Group 2, percent of notes linked is high in Module 1 but drops to 38% in

the second modules. Percent of notes linked in combined groups is similarly low. These

measures point to the need for more attention to linking concepts and the need for greater support

for participants to encourage this type of activity. The importance of build-on concepts will be

made explicit in the introductory session to Knowledge Building and by facilitator modelling

online.

Table 7

Online Activity Measures by Group, 2004/2005

Number of notes

created

Percent of notes read

Percent of notes

linked

Modules Group 1 Group 2 Group 1 Group 2 Group 1 Group 2

Mr. Singh’s Pain, Part 1 61 30 83 75 49 72

Mr. Singh’s Pain, Part 2 35 22 74 76 46 38

Mary’s Misery 43 75 32

Judy’s Last Days, Part 1 86 85 48

Judy’s Last Days, Part 2 29 75 19

Note. Group 1 (8 students +1 facilitator) n = 9; Group 2 (11 students + 1 facilitator) n = 12. Amalgamated groups:

Mary’s Misery (16 students + 2 facilitators) n = 18; Judy Part 1 (14 students + 2 facilitators) n = 16; Judy Part 2 (13

students + 2 facilitators) n = 15.

It is recommended that the Analytic Toolkit data be used for concurrent, embedded feedback in

the future. In this study, the Analytic Toolkit was used by the researcher to determine individual

continuing professional development credits for participants, for iterative program design, and

for data analysis.

83

4.2.4 Pilot Study, 2004/2005: Summary and Iterative Design Recommendations

In summary, aggregate scores of pre-/posttests demonstrated a positive increase of 11%

knowledge gain in pain understanding, a strong effect size of .89, high levels of online

participation in Knowledge Forum, and overall program satisfaction. However, despite these

positive results, the range, particularly on opioid scores on the pain posttest, was low. It appeared

based on item analysis that the low range of scores in this area was due to confusion about

titration and lack of knowledge in that area. The issue was not one of mathematics but instead

one of agreement on titration conversion scales and as the facilitators indicated, developing an

understanding of the art as opposed to the science of use of these medications, and how each

patient reacts differently. This is a key issue and all facilitators agreed that this problem should

be addressed with the addition conversion charts embedded in the case study and online tests,

and through more focused discourse in Knowledge Forum. Problematic questions were changed

on both pre- and posttests.

Contrary to common summative evaluation purposes, pre-/posttest in this program was used

primarily for formative self-assessment and iterative design of the program. In our model each of

the 32 questions was categorized and scored by objective. At completion of the pain pretest

participants were provided with a score card sorted by objective, providing quantitative feedback

information and recommended resources for addressing knowledge gaps and/or further learning.

Upon completion of the posttest participants were given the correct answers to all questions, in

depth explanations for each answer and further references. Novel to the project, learners were

then given an individual comparative score card, comparing their pre- and posttest scores by

objective and total scores, to help them self-assess change in knowledge and identify areas

weakness to scaffold continued improvement. Further details can be found elsewhere (Lax,

Singh, Scardamalia, & Librach, 2006).

Many studies have concluded that it is difficult for individuals to accurately assess their strengths

and weaknesses, or identify their own deficiencies (Eva & Regehr, 2005; Regehr, Hodges,

Tiberius et al., 1996). An important objective of this pilot study was to address this issue through

a novel means of feedback to support more accurate self-assessment. The ideal of “lifelong

innovativeness” (Scardamalia & Bereiter, 2005) is rooted in the notion of continuous

improvement. This pilot study provides an encouraging model of collective Knowledge Building

84

and scaffolded self-assessment for continuing medical education and was refined based on

feedback received for future years.

85

CHAPTER 5

RESULTS—CLUSTER 1:

TRADITIONAL OUTCOME MEASURES

5.1 Introduction

A pilot study was used to inform this four-year research study. This study was guided by the

same, but expanded, main research question (Table 8) and an additional series of subquestions

(Table 9).

Subquestions were organized in to three clusters, related to: (a) traditional outcome measures; (b)

performance over and above traditional measures (i.e., beyond learning as traditionally

conceived and measured); and (c) questions regarding the sociocognitive dynamics that enable

work over and above traditional learning. Results of these each of these clusters are presented in

three results chapters: Chapters 5, 6, and 7.

Table 4 in Chapter 3 provides a map of related elements organized by clusters of research

subquestions and related data collection and analyses used to obtain results presented here and in

the following two chapters. Extracts from that table will be used in the introduction of all results

chapters to provide an overview of the contents of each (see Tables 8 and 9).

Table 8

Research Question

Research Question


palliative care in a Web-based continuing medical education course, and if so, what social

network structural relationships and sociocognitive dynamics support knowledge

improvement and contribute to democratization and metadesign of Knowledge Building in

continuing medical education?

Note. Extracted from Table 4, Chapter 3.

86

Table 9

Cluster 1: Traditional Measures


Subquestion 1 Did students’ pain knowledge improve from pre- to posttest

and if so, is this improvement significant and what is the

effect size?


2-Tailed t-test Matched pre-/posttests

(4 years)

Cohen’s d Effect Size Matched pre-/posttests

(4 years)

Subquestion 2 Traditionally continuing medical education is a self-

directed or didactic experience; what are participants’

attitudes and opinions toward collaborative online

Knowledge Building in the End-of-Life Care Program?


Likert and Yes/No

responses

Online attitudes and

opinion survey (4 years)


Chapter 5 presents the results of the first cluster of subquestions related to traditional outcome

measures of learning, including knowledge gains based on pre-/posttest analysis, effect size, and

feedback on attitudes and opinion of their online experience. Results for participants the initial

year of the End-of-Life Care Distance Education Program, 2004/2005 were presented separately

from the following four years, 2005–2009, since questions on the pre- and posttest were changed

during the process of validation.

5.2 Pain Knowledge Pre-/Posttest Results by Year

and Cumulatively Across Four Years

Mean pre- and posttest scores were calculated for each of the four objectives, as well as the mean

difference, standard deviation, and effect size for each program year and cumulatively across

five years: 2005/2006, 2006/2007, 2007/2008, 2008/2009. A paired t-test was used to determine

statistical change across all four pain learning objectives. Cohen’s d statistic was used to measure

effect size.

87

5.2.1 Pain Knowledge 2005/2006 Pre-/Posttest Matched Results

Eleven of 15 participants completed both pre- and posttests in 2005/2006 program. Total mean

score across all Objectives (1 to 4) demonstrated a significant improvement in pain knowledge of

8%, from 73% on pretest to 81% on posttest, on paired t-test = 3.48, p < .006. Cohen’s d

demonstrated a large effect size of .87 related to overall knowledge improvement across the three

EoL Care pain modules (Table 10).

Objective 1, Pain Management, showed high scores on pretest, 89%, and on posttest, 92%,

indicative of a 3% mean difference. There was not much room for improvement, based on a high

level of knowledge in this area and the retention of easy questions in this section of the test.

Table 10

Matched Results of Pain Pre-/Posttests, 2005/2006

2005/2006 (n = 11)

Objective Pre M (SD) 95% CI Post M (SD) 95% CI t(10) p d

1. Pain Management 0.89 (0.11) [0.82, 0.96] 0.92 (0.14) [0.84, 1.01] 0.69 .506 0.27

2. Opioid Use 0.69 (0.16) [0.59, 0.78] 0.77 (0.12) [0.70, 0.85] 2.09 .063 0.56

3. Neuropathic Pain 0.71 (0.16) [0.61, 0.81] 0.76 (0.13) [0.68, 0.84] 0.88 .400 0.33

4. Bone Pain 0.76 (0.17) [0.65, 0.86] 0.87 (0.04) [0.84, 0.90] 2.06 .067 0.65

Total 0.73 (0.09) [0.68, 0.79] 0.81 (0.07) [0.77, 0.86] 3.48 .006 0.87

Objective 2, Opioid Use, showed improvement from 68% on pretest to 77% on posttest,

demonstrating mean difference of 9%. In comparison to scores from the previous year (40% on

pretest and 47% on posttest), the 2005/2006 range on pre- and posttest improved with the

addition of online conversion charts. However, content experts (program developer and

facilitator) remained concerned about the low level of knowledge around the use of opioids as

indicated on pretest. Hence, this area was identified as a knowledge gap/lack and associated

misconceptions became a focal point for facilitated, collective, online knowledge building

discourse.

Results from Objective 3, Neuropathic Pain, presented small positive change; the mean score

rose 5%, from 71% on the pretest to 76% on the posttest, which was considered low by context

experts. Pretest results were noted for further knowledge building in the collective online

discourse.

88

Results from Objective 4, Bone Pain, demonstrated a positive increase of 11% in mean score,

from 76% on the pretest to 87% on the posttest. Analysis at the question/answer level indicated

that change was related to use of terminology for identification of physiological concepts around

bone pain.

Results of item analysis were employed to ascertain effect of questions changed from the pilot

study. Items were scored and McNemar tests computed to assess whether the difference in

proportion correct was significant. The one-tailed tests should be interpreted with the usual .05

level of significance. Items with positive pre-/posttest change and no or negative pre-/posttest

change are included in Appendix E. Questions in relation to Objective 2, use of opioids, in

particular were reviewed to determine if these questions remained problematic even with the

inclusion of a titration chart in the 2005/2006 program. Specifically the opioid (part question

10_3 remained problematic and was targeted for revision. Other questions identified for review

and revision were pain management, question 17; opioid questions, 7, 20, and 33_4; neuropathic

pain questions 32_3 and 23_2 as demonstrated on the graph indicating negative difference from

pre- to posttest. These questions were targeted for revision for the 2006/2007 program.


Seven of 11 participants completed both tests in 2006/2007 (Table 11). Total mean score across

objectives 1 to 4 on pretest was 68% and on posttest was 78%. Improvement from pre- to

posttest was 10%.

Table 11

Matched Results of Pain Pre-/Posttests, 2006/2007

2006/2007 (n = 7)

Objective Pre M (SD) 95% CI Post M (SD) 95% CI t(6) p d

1. Pain Management 0.88 (0.13) [0.78, 0.98] 0.95 (0.08) [0.89, 1.02] 1.44 .200 0.57

2. Opioid Use 0.63 (0.17) [0.50 ,0.76] 0.73 (0.12) [0.63, 0.82] 1.46 .195 0.60


4. Bone Pain 0.70 (0.15) [0.58, 0.82] 0.86 (0.11) [0.77, 0.94] 2.71 .035 1.04

Total 0.68 (0.11) [0.60, 0.77] 0.78 (0.07) [0.73, 0.84] 2.15 .075 0.93

However, results on t-test were not significant, most likely due to small sample size. Analysis of

effect size demonstrated a large effect of 0.93 related to the three pain modules.

89


Twenty-six participants completed both the pain pre- and posttests in 2007/2008. Total mean


7%, from 68% on pretest to 75% on posttest, and on paired t-test = 2.22, p < .04. Cohen’s d

indicated a medium effect size of .64 related to overall knowledge improvement across the three

EoL Care pain modules (Table 12).

Table 12

Matched Results of the 2007/2008 Pain Pre-/Posttests

2007/2008 (n = 26)

Objective Pre M (SD) 95%CI Post M (SD) 95% CI t(25) p d

1. Pain Management 0.85 (0.20) [0.77, 0.93] 0.89 (0.19) [0.81, 0.97] 0.88 .387 0.20

2. Opioid Use 0.66 (0.15) [0.59, 0.72] 0.72 (0.19) [0.65, 0.80] 1.80 .085 0.44


4. Bone Pain 0.66 (0.20) [0.58, 0.74] 0.77 (0.25) [0.67, 0.88] 2.37 .026 0.56

Total 0.68 (0.11) [0.63, 0.72] 0.75 (0.15) [0.69, 0.81] 2.22 .036 0.64


Seventeen participants completed both the pain pre- and posttests in 2008/2009. Total mean


14%, from 67% on pretest to 81% on posttest, and on paired t-test = 4.30, p < .001. Cohen’s d

indicated a very large effect size of 1.15 related to overall knowledge improvement across the

three EoL Care pain modules (Table 13).

Table 13

Matched Results of the 2008/2009 Pain Pre-/Posttests

2008/2009 (n = 17)

Objective Pre M (SD) 95% CI Post M (SD) 95% CI t (16) p d

1. Pain Management 0.78 (0.20) [0.68, 0.89] 0.80 (0.19) [0.71, 0.90] 0.49 .632 0.10

2. Opioid Use 0.63 (0.16) [0.54, 0.71] 0.79 (0.15) [0.72, 0.87] 3.17 .006 1.00


4. Bone Pain 0.69 (0.18) [0.59, 0.78] 0.87 (0.08) [0.83, 0.91] 4.10 .001 1.42

Total 0.67 (0.12) [0.61, 0.73] 0.81 (0.10) [0.76, 0.86] 4.30 .001 1.15

90

5.2.5 Cumulative Matched Results of 2005–2009 Pain Knowledge Pre-/Posttests

Sixty-one participants completed both pain pre- and posttests across the four years analyzed,

2005 to 2009. Cumulative total mean score across all Objectives (1 to 4) demonstrated a

significant improvement in pain knowledge of 9%, from 69% on pretest to 78% on posttest, and

on paired t-test = 5.34, p < .001. Cohen’s d indicated an overall large effect size of .82 across all

years (Table 14).

Table 14

Cumulative Matched Results of the 2005–2009 Pain Pre-/Posttests

2005–2009 (N = 61)

Objective Pre M (SD) 95% CI Post M (SD) 95% CI t (60) p d

1. Pain Management 0.84 (0.18) [0.80, 0.89] 0.88 (0.17) [0.84, 0.93] 1.52 .135 0.20

2. Opioid Use 0.65 (0.16) [0.61, 0.69] 0.75 (0.16) [0.71, 0.79] 4.23 .001 0.65


4. Bone Pain 0.69 (0.18) [0.64, 0.74] 0.83 (0.18) [0.78, 0.87] 5.22 .001 0.76

Total 0.69 (0.11) [0.66, 0.72] 0.78 (0.12) [0.75, 0.81] 5.34 .001 0.82

As demonstrated by year and across all years, participants’ knowledge improved in areas

important to the practice of palliative care, including pain management, opioid use, neuropathic

pain, and bone pain. Some areas such as opioid use were seen to achieve the largest gains.

Understanding concepts about bone pain improved along with the vocabulary used, as

demonstrated from pre- to posttest. Concepts in pain management showed little improve due to

ceiling effect. Questions around these concepts were maintained to highlight the importance of

this area and reaffirm previously acquired knowledge.

5.2.6 Summary of Results of Pain Knowledge Pre-/Posttests

The pain knowledge pretest was conducted online before the course and the posttest was

conducted after three months of student participation in three online cases/modules. Analysis of

matched pre-/posttests were used to determine the results of our first research subquestion: Did

students’ pain knowledge improve from pre- to posttest and if so, is this improvement significant

and what is the effect size? In answer to this question, the analyses herein clearly demonstrate

total significant improvement from pretest to posttest across all years, with variability between

objectives. The 2008/2009 program demonstrated a particularly strong effect size of 1.15 while

the 2007/2008 program demonstrated the weakest effect size (0.64). Although incoming pretest

91

participant scores were similar (67% and 68% respectively) that was a substantial difference in

posttest scores (75% in 2007/2008, and 81% in 2008/2009), explaining difference in effect size.

Despite this range cumulative effective size across all years was 0.82; according to the definition

of Cohen’s d, anything over 0.80 is considered large.

5.3 Attitude and Opinion Survey Results 2005–2009

Participant attitudes and opinions on collaborative knowledge building and program design were

collected online at the end of the program. Demographic, Yes/No, Likert scale, and open-ended

questions were used in this survey. Data were collected all four years. Since attitudes and

opinions across years were similar, a summary of results from 2005–2009 programs is provided

here.

Exceptionally positive results were reflected on all aspects of content, collaborative KB,

relevance, and overall program rating. Sixty of 73 (82%) participants responded to the Attitudes

and Opinions Survey over four years, 2005–2009 (Table 15). The cumulative response rate was

82%. Of the 73, 56 were family medicine practitioners; three were specialists; and one did not

respond; 35 indicated they were male, 24 were female, and one person did not respond. Sixty

percent indicated they had been in practice more than 16 years. Forty-four percent were in

practice in Toronto, 22% in York region, 27% in Simcoe county, and the remainder were from

other areas in southern Ontario.

Table 15

Percentaged Results of Attitude and Opinions Summative Survey, 2005–2009

Demographics %

1. Location Toronto 43.8

York 21.7

Simcoe 26.7

Other 7.8

2. Gender Male 58.3

Female 40.0

n/a 1.6

3. Specialization Family medicine 93.3

Other/missing 6.7

(table continues)

92

Table 15 (continued)

Perceptions (%)

Strongly

disagree

Disagree

Equivocal

Agree

Strongly

agree

4. The goals and objectives were clearly

stated. 0 3.3 1.7 26.7 68.3

5. The goals and objectives were met. 0 1.7 3.3 36.7 58.3

6. The program is relevant to my practice. 1.7 3.3 8.3 28.3 58.3

7. My practice behaviour will change as a

result of this program. 0 5.0 8.3 18.3 68.3

8. The program was well organized. 1.7 0 3.3 21.7 73.3

9. Sufficient time was allotted for each

module. 0 1.7 6.7 13.3 78.3

10. The program was able to cover the

breadth of issues in palliative and end-of-

life care.

0 1.7 0 31.7 66.7

11. Collaborative e-learning was an enjoyable

way of learning. 5.0 3.3 8.3 31.7 51.7

12. The program satisfied my expectations. 0 1.7 11.7 35.0 51.7

Evaluation of objectives

Yes No n/a

13. The personal, social, cultural attitudes towards death and dying. 98.3 1.7

14. The current definition and the basic principles of palliative care. 100 0

15. The ethical issues confronting dying patients, their families and their

physicians including end of life decision-making, advanced

directives, euthanasia and assisted suicide.

95.0 5.0

16. The physical, psychological, social and spiritual issues of dying

patients and their families. 100 0

17. A systematic approach to working with families of dying patients. 98.3 1.7

18. Effective pain management. 100 0

19. The management of other physical symptoms (e.g. dyspnea,

constipation, skin care, mouth care, nausea and vomiting). 100 0

20. The identification and management of psychological issues for

patients. 98.3 1.7

21. The skills in providing psychological support and educational

counseling to dying patients and their families. 93.3 6.7

22. The roles of other disciplines in providing palliative care. 93.3 6.7

23. The community resources available to support patients in their

homes. 93.3 6.7

24. The approach to the last hours of caring in the home. 100 0

25. Incorporated evidence decision-making in caring for dying patients

and their families. 95.0 3.3 1.7

Continuing education feedback

26. Did you find this continuing education experience useful? 98.3 1.7

27. Did you acquire new knowledge? 98.3 1.7

28. Do you think as a result of this course you will change your approach

to palliative care? 86.7 13.3

(table continues)

93


Yes No n/a

e-Learning

29. I have taken a course using e-learning (i.e. online collaborative

learning) courseware before this one. 6.7 93.3

30. My Knowledge Forum (KF) group size was just right. 93.3 6.7

31. I found contributing discourse notes in KF quick to learn and easy to

use. 91.7 8.3

32. Digital resources in My.library were easy to access. 86.7 13.0

33. Digital resources in my library were helpful. 98.3 1.7

34. Test Yourself questions were helpful. 95.0 5.0

35. Pre- and posttest feedback was helpful. 98.3 1.7

36. Pre- and posttest feedback helped me determine my learning needs. 95.0 5.0

Collaborative Knowledge Building outcomes

37. Collaborative knowledge building discourse in KF helped me reflect

on new knowledge I gained. 90.0 6.7

38. Collaborative knowledge building discourse in KF helped me reflect

on different perspective presented. 95.0 5.0

39. Collaborative knowledge building helped me to become more aware

of my colleagues’ knowledge, attitudes and opinions. 98.3 1.7

40. Collaborative knowledge building helped me to identify changes I

would like to make in my practice. 88.3 11.7

Individual Reflection and Reflection-on-Practice sections were helpful. 85.0 10.0 5.0

41. Clinical summary notes by the Group Reporters were helpful. 91.7 6.7 1.7

42. Summative notes by the Discourse Analyst were helpful. 95.0 5.0

43. Facilitator feedback during discussion was helpful. 98.3 1.7

44. Facilitator feedback on final clinical summary note was helpful. 98.3 1.7

Perceived barriers and advantages

Strongly

disagree

Disagree

Equivocal

Agree

Strongly

agree

45. A barrier to e-learning in KF is the amount of

time required to read, contribute and respond in

writing.

13.3 25.0 21.7 30.0 10.0

46. Lack of convenient computer and Internet access

is a barrier to my e-learning.

50.0 18.3 6.7 16.7 8.3

47. An advantage of e-learning is that the group

work can be done asynchronously, at any time

and from anywhere, with a computer/ Internet

connection.

3.3 0.0 1.7 21.7 73.3

48. The face-to-face sessions (videoconferences)

were an important complement to e-learning.

3.3 1.7 15.0 28.3 51.7

(table continues)

94

Table 15 (concluded)

Overall evaluation

49. I would like to see more distance education

programs for Continuing Medical Education

made available.

3.3 0.0 5.0 35.0 56.7

50. I would like to see more distance education

programs for interprofessional Continuing

Education made available.

3.3 3.3 13.3 26.7 53.3

51. Based on my experience of collaborative e-

learning, I would recommend it to a colleague.

5.0 1.7 6.7 25.0 61.7

Poor

Below

average

Average

Above

average

Excellent

52. Overall, I would rate the online collaborative

knowledge building component of this program

as:

0.0 6.7 6.7 26.7 60.0

53. Overall, I would rate the face-to-face sessions

(video-conferences) of this program as:

1.7 1.7 15.0 36.7 45.0

Note. n = 60 of 73 respondents (82%).

Four respondents had taken an e-Learning course previously; 56 (93.3%) respondents had not.

Despite this, most (91.7%) participants found contributing discourse notes in Knowledge Forum

quick to learn and easy to use.

In response to a series of Yes/No questions the following attitudes and opinions were found.

Almost all (98.3%) physician participants indicated their CME experience was useful; they

gained new knowledge (98.3%) and as a result, they would make changes in their practice

(86.7%). Most (95%) respondents indicated collaborative KB helped them reflect on different

perspectives and understand their colleagues’ perspectives (95%). Ninety percent of respondents

indicated that collaborative knowledge building discourse in Knowledge Forum helped them to

reflect on new knowledge. All most all respondents (98.3%) found facilitator feedback during

discussion helpful. Pre-/posttest embedded feedback was considered helpful by 95% of

respondents, who also indicated that this type of structured, embedded, feedback assisted them

determine their learning needs.

A 5-point Likert scale was used to more discriminately determine participants’ attitudes and

opinions on online knowledge building. Barriers to online learning are perceived by less than

half of the participants. Forty percent agreed/strongly agreed that the amount of time to read and

contribute/write online notes was a barrier. Very few (15%) agreed/strongly agreed that

computer and Internet access was a barrier. Most considered (95% agree/strongly agree) that the

95

advantage of online learning was that group work can be done asynchronously. Ninety-two

percent would like to see more distance education program for CME and 86.7% would

recommend online collaborative learning to a colleague, based on their experience in this course.

The two videoconferencing sessions (the opening Introduction to Palliative Care session and the

Symptoms Other Than Pain session) were highly rated; 81.7% of respondents rated these

sessions as above average/excellent. The online collaborative knowledge building component

was rated above average by 26.7% of respondents and excellent by 60% of respondents, totaling

86.7%.

In conclusion, although the amount of time required to read, write and contribute to online

collaborative discourse may be considered a barrier, the overall strong results provide an

overwhelming picture of satisfaction with collaborative knowledge building. Over 91% indicated

they would like to see more continuing medical education programs made available online.

In summary, survey results indicated that online collaborative knowledge building helped

participants identify their strengths, weaknesses and knowledge gaps for further cognitive work.

Reflection on different perspectives within the discourse and facilitator feedback scaffolded

knowledge building. The collective discourse was highly rated by participants and can be seen as

a form of unstructured, formative feedback that is learner-driven and emergent. The use of

pre-/posttests was also determined to be helpful to participants in identifying personal areas of

strength and weakness, and knowledge lacks. The embedded pre-/posttests created a system of

structured, formative feedback, based on standard clinician/expert knowledge expectations of

what one should know to competently practice in the field. Thus both structured and unstructured

embedded, concurrent assessments were perceived positively by participants.

Aggregate scores of the 2005–2009 pre-/posttests demonstrated strong knowledge gains in pain

understanding. These strong results appear to be related to overall positive attitudes and opinions

toward collaborative knowledge building and embedded formative feedback, as well as to

findings of high levels of activity and interactivity in online discourse, the results of which will

be presented next.

96

5.4 Summary of Traditional Outcome Measures: 2005–2009 Results of Pain Knowledge

Pre-/Posttest Results and Attitude and Opinion Survey Results

Both the pain knowledge pre-/posttest scores and the participant Attitudes and Opinions Survey

feedback demonstrated strong results (in answer to our second research subquestion). These

positive traditional individual aggregate outcome measures can be used to validate the

educational “success” of this program. However, establishing that the program “succeeded” by

improving individual knowledge and satisfaction, is very different from understanding how

individuals participated and interacted to improve knowledge within a social-mediated

environment, and if knowledge work in design mode emerged.

In the next chapter, Knowledge Forum participation and interaction patterns will be analyzed to

provide us with a window into this aspect. Results of various social network analyses will also be

presented to describe individual and collective performance over and above traditional learning

outcomes.

97

CHAPTER 6


PERFORMANCE OVER AND ABOVE TRADITIONAL MEASURES

6.1 Introduction

Chapter 6 presents the results of the second cluster of research subquestions, numbers 3, 4, and 5

(see Table 16), which describe performance over and above traditional learning measures. As

indicated previously, subquestions are organized in to three clusters related to: (a) traditional

outcome measures, (b) performance “over and above” traditional measures (beyond learning as

traditionally conceived and measured), and (c) questions regarding the sociocognitive dynamics

that enable work over and above traditional learning.

Performance “over and above” traditional learning measures based on results of analyses

performed using Knowledge Forum’s suite of analytic tools and Netminer®

software for social

network analysis are presented in this chapter.

Knowledge Forum Analytic Toolkit results of read/write activity, build-on interactivity, across

all years are shown. The final year, 2008/2009, that demonstrated largest knowledge gains was

selected for further in-depth analyses; graphic contribution and social network analysis tools in

Knowledge Forum’s Analytic Toolkit were used to determine read and build-on network

characteristics. Additional results of numerous social network structural Analyses are presented

herein, as well as analyses of network cohesion and cliques.

Table 16

Cluster 2: Beyond Traditional Measures

CLUSTER 2 PERFORMANCE OVER AND ABOVE TRADITIONAL MEASURES

Subquestion 3 What are participants’ online activity and interactivity measures, by module, by group,

and by year, in the 2005–2009 study?


ATK tools in Knowledge Forum 5 online modules each year (Pilot

Study and 4 years)

SNA tools in Knowledge Forum 5 online modules each year

(2008/2009)

(table continues)

98



pre- to posttest and is there a significant difference between groups?

Data analysis Data sources

2-Way ANOVA 2008/2009 Groups 1 and 2 pre-

/posttests


greater in Group 1 than in Group 2. What are the social network structural differences

between groups that supported increased knowledge improvement and how are these

differences related to centralization and/or democratization of participation and ideas?

Data analysis Data sources

Structural SNA Network edges (measures and visualizations) 2008/09 Groups 1 and 2; 10 modules

in total; Build-ons and notes read

(with/without facilitator); = 20 total

Network density (measures and visualizations) as above

Eigenvector centrality (measures and

visualizations)

as above

In-degree centrality (measures and visualizations) as above

Out-degree centrality (measures and

visualizations)

as above

Cohesion index (measures and visualizations) 2008/2009 Groups 1 and 2; 10

modules in total; Build-on notes,

with facilitator

Cliques (measures: # of cliques; # of members;

members; maps)

as above

Relationship of

SNA to Knowledge

Improvement

t-test for significant difference and effect size

(comparison of SNA of build-ons between groups,

across 3 pain modules)

2008/2009 Groups 1 and 2; 3 pain

modules; social network measures of

build-on notes

Spearman correlation (of posttest scores with

social network variables)

2008/2009 Groups 1 and 2; pain

posttest scores; social network

measures of build-on notes


6.2 Online Activity and Interactivity Measures Results

Knowledge Forum Analytic Toolkit (ATK) measures of individual and collective read and write

activity were calculated, as well as online interactivity build-on measures, by year for all four

years of the program, across all five online modules: Mr. Singh’s Pain, Parts 1 and 2, on basic

pain management; Mary’s Misery on complex pain management; and Judy’s Last Days, Parts 1

and 2, dealing with the last days of life. Participants worked in each of the five modules for one

month each. Individual and collective results were posted online for participant formative

feedback and self-monitoring purposes. These statistics were also used to provide individual

activity “report cards” for Mainpro continuing education credits at the end of the course.

99

6.2.1 Results of 2005–2009 Read, Write, Build-On Measures

Knowledge Forum ATK results demonstrated particularly strong levels of online read/write

activity within all five modules and across four years, in the 2005/2006, 2006/2007, 2007/2008

(Table 17) and 2008/2009 programs (Table 18). ATK measures indicated:

· in 2005/2006 participants read a mean of 86.3% of notes across all modules and

wrote an average of 18.67 notes per module;

· in 2006/2007 participants read a mean of 92.3% of notes across all modules and


· in 2007/2008 participants in Group 1 read a mean of 92.6% of notes and wrote an

average of 30.93 notes per module; participants in Group 2 read a mean of 92.4% of

notes across all modules and wrote an average of 18.25 notes per module;

· in 2008/2009 participants in Group 1 read a mean of 79.33% of notes and wrote an

average of 36.82 notes per module; while participants in Group 2 read a mean of

85.7% of notes across all modules and wrote an average of 25.1 notes per module.

In 2005/2006 participants created a total of 280 notes. In 2006/2007 participants contributed a

total of 369 notes in the same modules. In 2007/2008 participants were split into two groups. The

2007/2008 Group 1 participants contributed a total of 433 notes while Group 2 participants

contributed a total of 292 notes. In 2008/2009 the participants in Group 1 contributed 405 notes

and participants in Group 2 contributed a total of 251 notes. Since reading measures were

relatively high throughout, measures of note creation were deemed to be a focal point for further

analysis.

100

Table 17

Online Activity Measures from 2005/2006, 2006/2007, and 2007/2008

Number of notes

created Percentage of notes read

Online activity 2005/2006 (n = 15)

Mr. Singh’s Pain, Part 1 72 85


Mary’s Misery 48 86

Judy’s Last Days, Part 1 60 86


Total 280

Average/participant 18.7

M (SD) 86.0 (1.2)

Online activity 2006/2007 (n = 11)



Mary’s Misery 53 96



Total 369

Average/participant 33.6

M (SD) 93.6 (3.4)

Online activity 2007/2008

Number of notes created Percentage of notes read

Group 1 Group 2 Group 1 Group 2 Group 1(n) Group 2(n)

Mr. Singh’s Pain, Part 1 109 90 94.2 86.2 14 19


Mary’s Misery 88 82 93.0 95.8 15 16

Judy’s Last Days, Part 1 44 43 97.5 95.4 13 16


Total 433 292

Average/participant 30.93 18.25

M

(SD)

94.5

(3.0)

93.0

(4.0)

101

Table 18

Summary of Online Activity and Interactivity Measures, 2008/2009

Number of notes

created

Percentage of notes

read

Number of build-on

notes

Group 1 Group 2 Group 1 Group 2 Group 1 Group 2



Mary’s Misery 78 55 86.9 92.8 60 43



Total number 405 251 309 197

Average/participant 36.8 25.1 28.1 19.7

M 79.3 85.7

(SD) (4.8) (8.9)

Note. Group 1 (n = 11); Group 2 (n = 10)

Knowledge Building measure of number of build-on notes were not recorded for all years (other

than 2008/2009) since the number of continuing medical education Mainpro credits awarded by

The College of Family Physicians of Canada was only based on read/write participation statistics

in this program. Other Knowledge Building measures were available in the ATK (Burtis, 2001)

such as “who reads whose notes.” These statistics are interesting and informative but were not

used in this study. The ATK results table of “who reads whose notes” was particularly difficult

to interpret since results was presented numerically cross-referenced by individual to individual.

Network change over time was almost impossible to apprehend through numerical

representations of data. This issue was addressed by the recent creation of a number of

Knowledge Building graphic tools in Knowledge Forum, that were employed to obtain in-depth

results of the 2008/2009 online activity and interactivity and relationships across dimensions,

presented in the following sections.

In conclusion, ATK results indicated a high level of activity each year, in regards, to number of

notes created and percentage of notes read online. Online activity was almost always strongest in

the first two modules on pain. Understanding opioid use was the key objective of modules 1 and

2 and understanding neuropathic pain was key to modules 1, 2, and 3. Activity was particularly

strong in 2008/2009. Closer examination of 2008/2009 groups indicated that the total number of

notes created by Group 1 (n = 11) was substantially higher than Group 2 (n = 10). In 2008/2009

102

Group 1 created 154 more notes than Group 2; on average, more than 10 notes per participant. It

is particularly important to note that Group 1 created 112 more build-on notes than Group 2.

Percent of notes read was high across both groups. Further investigation of the 2008/2009 dataset

will be done since this was the highest performing year. Relationships across dimensions and

between groups will be examined.

6.3 Results of 2008/2009 Online Performance and Social

Network Relationship Measures (Beyond Learning)

The End-of-Life Care Distance Education Program was transferred to a updated version of

Knowledge Forum that provided access to a new graphical ATK in 2008/2009. It was intended

not only to support research, but to provide participants with immediate access to up-to-date

assessments for formative feedback. Concurrent and embedded assessments were represented

graphically and statistically, as opposed to previous ATK numeric only measures. The goal was

to turn assessment over to participants, for structured feedback on their current and past

performance, fostering reflection, scaffolding self-assessment, and supporting higher-level

metacognitive work aimed at improving individual and collaborative knowledge building. By

providing participants access to assessment, it no longer is in the sole purvey of facilitators,

teachers, and/or researchers, and can be considered a step towards democratization of the

pedagogic environment. The updated ATK includes various contribution/activity measures,

social network analysis, semantic overlap, and other writing and language assessment tools

(Teplovs, 2008).

Online contribution and social activity network measures were used in this study to analyze the

Knowledge Forum discourse/design space of the 2008/2009 EoL Care Program. The graphic

contribution tool was helpful in performing comparative analyses between facilitator and student

contributions, specifically on measures of total note written, number of group notes read, and

number of build-on notes. These graphic measures were converted back to numeric measures for

consistency of representation and ease of comparison as indicated in Table 16 and included

below to present facilitator/student comparative results. In addition, the results of two social

network interactivity measures are presented below. They are “who built on whose notes” and

“who read whose notes.” Patterns of activity and interactivity are compared. These new tools

provided new types of information for analysis and understanding these social network

relationships for Knowledge Building.

103

6.3.1 Group 1, 2008/2009 ATK and Social Network Assessment Measures

Social network analysis results of each KF module, that took place over a one-month time

period, across two groups of participants (each moderated by one palliative care

expert/facilitator) are presented below. Five online modules—Mr. Singh’s Pain, Parts 1 and 2, on

basic pain management; Mary’s Misery, on complex pain management; and Judy’s Last Days,

Parts 1 and 2, dealing with the last days of life—were analyzed. Note contributions, build-ons,

and reading measures are presented graphically by participant that is particularly helpful for

student-to-student comparatives and students to facilitator comparatives. Results of social

network analysis of who built on whose notes and who read whose notes are presented in terms

of network edge measures and distribution patterns.

6.3.1.1 Group 1, Mr. Singh’s Pain, Part 1. Contribution results of analysis of the first pain

module for Group 1 demonstrated a high level of activity by one facilitator and nine participants

(Figures 1, 2, 4). Although the facilitator contributed approximately twice as many notes as other

participants, the number of notes read was more evenly distributed across all participants.

Social network analysis of build-on activity reflects good distribution (Figure 3). Social network

analysis of who read whose notes represents strongly distributed activity; network edges

measured 110 (Figure 5).

104

Figure 1. Total note contributions Group 1, Mr. Singh’s Pain, Part 1, 2008/2009

Figure 2. Total number of build-on notes created Group 1, Mr. Singh’s Pain, Part 1, 2008/2009

105

Figure 3. Social network analysis of who built-on whose notes, Group 1, Mr. Singh’s Pain, Part

1, 2008/2009

Figure 4. Total number of notes read, Group 1, Mr. Singh’s Pain, Part 1, 2008/2009

106

Figure 5. Social network analysis of who read whose notes, Group 1, Mr. Singh’s Pain, Part 1,

2008/2009

6.3.1.2 Group 1, Mr. Singh’s Pain, Part 2. Results of discourse space analysis of the second

pain module for Group 1 contributions indicated 101 notes were created, over a one-month

period, by one facilitator and nine participants (Figure 6). Overall contributions, build-ons, and

reading of notes is strong (Figures 6, 7, 9). Facilitator activity is at least double that of most

participants, but relatively lower in comparison to previous module. Participant activity is more

evenly distributed than in the first module. Participants build-on each other’s notes, as well as the

facilitators notes.

Social network analysis of participant build-on activity reflects similar distribution to that

represented graphically (Figure 8). Again, group read activity was extremely high (Figure 9).

Social network analysis of who read whose notes represents distributed activity, with similar

network edges measuring 110 (Figure 10).

107

Figure 6. Total note contributions, Group 1, Mr. Singh’s Pain, Part 2, 2008/2009


108


2, 2008/2009


109


2008/2009

6.3.1.3 Group 1, Mary’s Misery, Pain Module 3. Results of discourse space analysis of the

third pain module, Mary’s Misery, for Group 1 indicated a total of 78 notes were created, over a

one-month period, by one facilitator and nine participants; build-on notes and reading were again

very strong (Figures 11, 12, 14).

Social network analysis of participant build-on activity reflects similar distribution; network

edges measured 21 (Figure 13). Facilitator contributions decreased in comparison to previous

modules. One particular participant’s activity has increased to become similar to that of the

facilitator. This is a positive transition indicating a shift in locus of power. Social network

analysis of who read whose notes, again, was distributed with network edges measuring 91

(Figure 15).

110

Figure 11. Total note contributions, Group 1, Mary’s Misery, 2008/2009

Figure 12. Build-on note contributions, Group 1, Mary’s Misery, 2008/2009

111


2008/2009

Figure 14. Total number of notes read, Group 1, Mary’s Misery, 2008/2009

112


2008/2009

6.3.1.4 Group 1, Judy’s Last Days, Part 1. Contribution results of discourse space analysis of

the fourth online module, Judy’s Last Days, Part 1, for Group 1 indicated 65 notes were created,

over a one-month period, by one facilitator and nine participants (Figure 16). Facilitator

contributions and build-ons were twice as high in comparison to other participants (Figures 16,

17); similar to other modules, reading results were evenly distributed (Figure 19).

Social network analysis of participant build-on activity reflects similar distribution and network

edges measured 14 (Figure 18) Social network analysis of who read whose notes, again, was

distributed with network edges measuring 76 (Figure 20), a decrease in comparison to the

previous module. This is probably due to decrease in number of network participants.

113

Figure 16. Total note contributions, Group 1, Judy’s Last Days, Part 1, 2008/2009

Figure 17. Build-on note contributions, Group 1, Judy’s Last Days, Part 1, 2008/2009

114

Figure 18. Social network analysis of who built-on whose notes, Group 1, Judy’s Last Days, Part

1, 2008/2009

Figure 19. Total number of notes read, Group 1, Judy’s Last Days, Part 1, 2008/2009

115

Figure 20. Social network analysis of who read whose notes, Group 1, Judy’s Last Days, Part 1,

2008/2009

6.3.1.5 Group 1, Judy’s Last Days, Part 2. Contribution results of discourse space analysis of

the fifth online module, Judy’s Last Days, Part 2, for Group 1 indicated 65 notes were created,

over a one-month period, by one facilitator and nine participants (Figure 21). Forty-five build-on

notes were created (Figure 22). Note contributions and reading were distributed in comparison to

the previous module (Figures 21, 24); with reading results again reaching a very high level.

Social network analysis of participant build-on activity reflects similar distribution and increase

in network edges to 20 (Figure 23). Social network analysis of who read whose notes, again, was

distributed with network edges measuring 66 (Figure 25), a slight decrease in comparison to the

previous module.

116



117


2, 2008/2009


118


2008/2009

6.3.2 Group 2, 2008/2009 ATK and Social Network Assessment Measures

Group 2 discourse space contribution results and social network analysis of the same five

modules, Mr. Singh’s Pain, Parts 1 and 2, Mary’s Misery, and Judy’s Last Days, Parts 1 and 2,

were analyzed and are presented below.

6.3.2.1 Group 2, Mr. Singh’s Pain, Part 1. Contribution results of various analysis of the first

pain module, Mr. Singh’s Pain, Part 1 for Group 2 demonstrated a moderate level of activity.

Sixty-nine notes were contributed to the discourse over a one-month period, by one facilitator

and seven participants (Figure 26). Each participant created between 1 and 12 notes. The

facilitator contributed 26—more than twice as many as the most active participant. The number

of build-on notes created was 58 (Figure 27).

Social network analysis of participant build-on activity is represented below (Figure 28). The

dominance of the number of facilitator discourse notes can be readily seen in the pattern of

distribution; network edges measured 18. Read activity by the group was extremely high (Figure

119

29). Similar to Group 1, the facilitator contributed approximately twice as many notes as other

participants. The number of notes read was very high and evenly distributed across all

participants. Social network analysis of who read whose notes is represented through the

distribution of activity; network edges measure 67 (Figure 30).



120


1, 2008/2009


121


2008/2009

6.3.2.2 Group 2, Mr. Singh’s Pain, Part 2. Results of discourse space analysis of the second

pain module for Group 2 contributions indicated 60 notes were created, over a one-month period,

by one facilitator and eight participants (Figure 31). Facilitator activity was extremely high and

participant activity was relatively low. Number of build-on notes created was 48 (Figure 32).


edges increased to 21 (Figure 33). Group read activity was extremely high (Figure 34); most

participants read most notes. Facilitator contributions remained high in comparison to the

previous module. Similarly, social network analysis of who read whose notes represents

distributed activity. Network edges measured 55, similar to the previous module (Figure 35).

122



123


2, 2008/2009


124


2008/2009

6.3.2.3 Mary’s Misery, Pain Module 3, Group 2. Contribution results of discourse space

analysis of the third pain module, Mary’s Misery, for Group 2 indicated 55 notes were created,

over a one-month period, by one facilitator and seven participants (Figure 36). Facilitator activity

decreased in relationship to participant activity. Forty-three build-on notes were created (Figure

37). Social network analysis of participant build-on activity reflects similar distribution; network

edges decreased to 12 (Figure 38). Total read activity was strong (Figure 39). Social network

analysis of who read whose notes remained almost the same as the previous module and

demonstrated strong distribution; network edges measured 54 (Figure 40).

125

Figure 36. Total note contributions, Group 2, Mary’s Misery, 2008/2009

Figure 37. Build-on note contributions, Group 2, Mary’s Misery, 2008/2009

126


2008/2009

Figure 39. Total number of notes read, Group 2, Mary’s Misery, 2008/2009

127


2008/2009

6.3.2.4 Judy’s Last Days, Part 1, Group 2. Analysis of the fourth Group 2, online module,

Judy’s Last Days, Part 1, indicated that 36 notes were created, over a one-month period, by one

facilitator and four participants (Figure 41); 28 build-on notes were created (Figure 42).

Facilitator contributions were high in comparison to other participants. Social network analysis

of participant build-on activity reflects similar distribution; network edges measured 6 (Figure

43). Total read activity was satisfactory (Figure 44). Social network analysis of who read whose

notes, again, was distributed with network edges measuring 17 (Figure 45). More reading than

writing was event in this module. Twelve participants read the notes of others while only four

contributed.

128



129


1, 2008/2009


130


2008/2009

6.3.2.5 Judy’s Last Days, Part 2, Group 2. Contribution results of discourse space analysis of

the fifth Group 2, online module, Judy’s Last Days, Part 2, indicated 31 notes were created, over

a one-month period, by one facilitator and three participants (Figure 46). Twenty build-on notes

were created (Figure 47). Contributions were distributed in comparison to the previous module.


edges measured 7 (Figure 48). Total read activity was extremely high (Figure 49). Seven

participants, in addition to the facilitator read almost all notes. Social network analysis of who

read whose notes, again, was distributed with network edges measuring 31 (Figure 50), a

decrease in comparison to the previous module.

131



132


2, 2008/2009


133


2008/2009

6.3.3 Social Network Pattern Analysis Across Modules and Comparatively Between

Groups, 2008/2009

New KF graphical tools enable new types of data representation and demonstration of

relationships. Visually demonstrated below are balances and shifts in traditional student/teacher

power relationships, allowing us to see the democratization of a knowledge building

environment as participants assume greater responsibility in the discourse design space. The first

analysis below (Figures 51, 52, 53, 54, 55) is composed of the 2008/2009 Group 1 social

network analysis of who built on (blue visualizations) and who read whose notes (green

visualizations).

Significant shifts are seen in who built on whose notes between the first two modules. In Mr.

Singh’s Pain, Part 1 the dominant patterns are fan-shaped and arch-shaped. The fan-shape is

indicative of the strong converging focus on facilitator discourse and conversely facilitator

finger-like extensions to participants. The arch-shape are indicative of equally strong

interparticipant discourse. These strong, definitive fan and arch patterns change and become

134

more diffuse over each successive module, until we find strong patterns, voices of many

participants equal to or stronger than the facilitators’. By Judy’s Last Days, Part 2 the patterns of

interactivity are well distributed, indicative of a shift in power structure towards one that is

maintained by all. Review of database note contributions will verify expected balance between

belief- and design-mode work and emergent ideas based on ideas brought into the discourse by

participants. Patterns of distribution of who read whose notes (green visualization) remains

highly active and stable across the first three modules and weakens in the last two modules, due

to the reduced number of participants. Summative survey comments indicates drop-out rates in

the last two modules may be due to the length of the course, from October to April.

Figure 51. Group 1, Mr. Singh’s Pain, Part 1


135

Figure 53. Group 1, Mary’s Misery

Figure 54. Group 1, Judy’s Last Days, Part 1


Note. Figures 51–55: 2008/2009, Group 1, social network activity visualizations across all modules of who built on

whose notes (blue visualizations) and who read whose notes (green visualizations).

136

Group 2 social network visualization patterns were also analyzed to determine balance of power

and shifts across modules (Figures 56, 57, 58, 59, 60). The visual patterns for this group are

different from the first group. Discourse build-on notes in the first module are reasonably

distributed across facilitator and a few strong participants. Social network visualization of who

read whose notes is more active than build-on contributions. Therefore more participants are

reading than building-on. Although actual numbers of participants in Group 2 diminish in each

module as the program continues on, distribution of build-on notes remain quite evenly spread

across participants and the facilitator. In the final two modules one particular participant appears

to dominate, surpassing the facilitator, appearing to have taken on the strongest position in the

discourse, and potentially the role of the facilitator. Triangular patterns characters this visual

analysis, which is indicative of a non-centralized, distributed, and democratic environment,

primarily shared between two participants and the facilitator. Although both build-on and read

activity and interactivity appears weak, particularly in the final two modules, this may only be

due to the decrease in number of participants, not in the interest in ideas in the database. The

level of reading relative to the number of participants is high throughout.


137


Figure 58. Group 2, Mary’s Misery


138


Note. Figures 56–60: 2008/2009, Group 2, social network activity visualizations across all modules of who built on

whose notes (blue visualizations) and who read whose notes (green visualizations).

6.3.4 Summary of Social Network Analysis and Knowledge Forum Tools

The high level of sustained activity, by Group 1, throughout the five modules and over a five-

month period of online engagement is notable. Deep engagement, commitment to intentional

learning, and the development of a knowledge building community becomes apparent through

these visualizations of notes contributed, notes read, and social network relationships.

Social network patterns were different in Groups 1 and 2. The visual display of these results

made it easy to compare different relationships between participants and facilitator, changes

across time, and demonstrate comparative aspects between groups. It can be quickly understood

through comparative visual representation of data results that Group 2 with fewer participants

than Group 1 were not as active, but yet worked more democratically from the initial module,

which may reflect on both the facilitator and group participants. It would be reasonable to expect

that all groups will have their own characteristics and distribution patterns.

The Knowledge Building assessment visualization tools represent activity and interactivity in

new ways, beyond numbers, and thereby enable new types of analyses of patterns and

relationships across time, shifts in dynamics, and comparisons to be understood. These

visualizations provide compelling evidence in a new-found ways, important to research, the

advancement of ideas, and perhaps most profoundly for participant use for collective and self-

assessment, as embedded concurrent feedback.

139

However, the Knowledge Forum visualization tools and visual analyses, like statistical tools and

numerical analyses, have their limitations. They do not provide us with an understanding of the

discourse itself. For example, we do not know how participants worked with knowledge between

themselves or what role the facilitator played. Did the facilitator use a didactic or Socratic

approach? Did participants question each other? Did they cite readings from the digital library

and/or refer to their own experiences in practice? If misconceptions emerge in the discourse were

they dealt with by the collective, or by the facilitator, or both? Did participants contribute their

own emergent ideas and work in design-mode, as well as belief-mode? Answers to these

questions can be revealed through analysis of on the collective online discourse itself.

Participant ideas and understandings are made visible through their online note contributions.

Qualitative analysis of discourse within notes, between notes, and across notes, would enables us

to further understand important dimensions of online cognitive collective activity and

interactivity, how ideas and misconceptions are worked with, and other aspects of knowledge

building. Qualitative analysis of note content and latent semantic analyses are beyond the scope

of this thesis.

However, changes seen in network distribution patterns between facilitator and participants will

be verified through a review of the 2008/2009 Group 1 collective discourse notes in each

module. Social network distribution patterns across modules over time have enabled us to

determine how student/facilitator and student/student interactions changed across all five

modules. Results of more detailed social network analysis and a review of discourse notes will

help explain changes in patterns and verify interpretations of the current analysis to help us

unravel representations of design mode and belief mode co-creation of knowledge.

6.4 Results for Groups 1 and 2 of 2-Way ANOVA of 2008/2009

Pain Knowledge Pre-/Posttests

Consistent strong outcomes in knowledge improvement, effect, and online collective

activity/interactivity, as well as extremely positive attitudes of communal value of collective

engagement for online knowledge building in this CME course were demonstrated across all

years of this study. A closer examination of the social networks of different groups using the

same cases and comparison of physician/students and facilitator/palliative care experts

interactivity patterns is required to better understand the structural relationships that support

140

knowledge improvement. The 2008/2009 cohort year was chosen for detailed structural social

network analyses.

The 2008/2009 cohort was typical in that their mean pretest score was 67% (slightly lower but

similar to the previous 2-years). However, this two-group cohort demonstrated the largest

increase in pre-/posttest scores and effect size during the five years of this study and was

therefore, selected for further detailed social network structural and semantic analyses of

Knowledge Building relationships. Although clinical cases were the same and participant

demographics and facilitator demographics were very similar across years, online note creation

activity was substantially higher in Group 1 than in Group 2.

Therefore, in the next research subquestion, I asked: Is there a significant difference in

2008/2009 Group 1 and Group 2 pain knowledge scores from pre- to posttest and if so, is there a

significant difference between groups and effect size? A 2-way ANOVA was performed in SPSS

to determine if there is a significant difference from pre- to posttest by group and between

groups. Results are indicated in Table 19.

Overall, the difference between the 2008/2009 matched pre- and posttest means combined over

group was significant, F (1,16) = 17.97, p < .001. Overall difference between groups was not

significant. However, both groups demonstrated a similar strong effect size. Group 1

demonstrated a 16% improvement in total pain knowledge, from 68% on pretest to 84% on

posttest. In comparison Group 2 scores increased 12% from 66% on pretest to 78% on posttest.

These noteworthy differences in knowledge improvement between 2008/2009 Groups 1 and 2

demand further examination.

Table 19

Results of 2-way ANOVA Groups 1 and 2, 2008/2009

Pre Post

Groupa M (SD) 95% CI

M (SD) 95% CI Difference d

b

1 0.68 (0.14) [0.59, 0.77] 0.84 (0.12) [0.59, 0.76] 0.16 1.11

2 0.66 (0.11) [0.76, 0.92] 0.78 (0.09) [0.71, 0.86] 0.12 1.12

Note. The difference between the pre- and posttest means combined over group was significant, F (1,15) = 17.94,

p < .001. a Group 1, n = 8; Group 2, n = 9.

b Cohen’s measure of effect size.

141

Although both groups demonstrated statistically significant knowledge gains, Group 1 gains

were somewhat larger than those in Group 2. Cases and learning objectives were the same,

student incoming knowledge and facilitator content expertise were similar. Differences in

knowledge improvement between groups may be explained by further examination of collective

social network interactions, by distinguishing between facilitator and student patterns of

participation and work with ideas, as well as by detailed exploration of individual relationships

within the social network.

6.5 Social Network Structural Analysis Results 2008/2009

Pain knowledge improved significantly in both groups; however, knowledge increase was greater

in Group 1 than in Group 2, which lead us to the next and fifth research subquestion: What are

the social network structural differences between groups that supported increased knowledge

improvement and how are these differences related to centralization and/or democratization of

participation and ideas?

Investigation of this subquestion entailed comparative analyses of facilitator and student

interactivity across modules and relation to various knowledge building and social network

measures. Analyses were performed on 2008/2009 Group 1 and Group 2, with facilitator and

without facilitator (students only), and in relation to who built-on whose notes and who read

whose notes using Netminer 3 software.

The following structural social network analyses were conducted:

· Number of network links and network density;

· Social network centrality measures, specifically Eigenvector centrality, in-degree

centrality, and out-degree centrality;

· Clique members and clique cohesion index; and

· Clique characteristic, specifically total number of cliques, average size of cliques,

mean clique cohesion index, and number of cliques the facilitator belongs to.

Network position (centrality of facilitator/students and distribution of students) and relationship

to network power (prestige and dominance/democracy) although related to social network

structural analysis will be considered in a separate section and linked to sociocognitive dynamics

of the network.

142

6.5.1 Structural SNA: Network Links and Network Density Results

All social network analyses were conducted on both groups, once including the Facilitator (+F)

and once without the Facilitator (–F), students only. This analysis enabled us to distinguish

power relationships, such as facilitator dominance and student network strength (Tables 20, 21,

22).

Table 20

Number of Edges and Network Density of Build-on Notes With and Without Facilitator,

Group 1

Group 1 Who built-on whose notes Who read whose notes

Facilitator

inclusion

Number

of nodes

Number of

links/edges

Network

density

Number of

links/edges

Network

density

Mr. Singh’s Pain, Part 1

+F 11 38 0.345 93 0.845

–F 10 21 0.233 74 0.822


+F 11 38 0.336 110 1.000

–F 10 22 0.244 90 1.000

Mary’s Misery

+F 11 30 0.264 108 0.982

–F 10 17 0.189 88 0.978

Judy’s Last Days, Part 1

+F 11 24 0.191 65 0.591

–F 10 11 0.100 49 0.544


+F 11 28 0.255 63 0.573

–F 10 14 0.156 48 0.533

Total number of edges and mean network density (SD)

+F 158 0.278 (0.06) 439 0.798

(0.21)

–F 75 0.184 (0.59) 349 0.783

(0.22)

Note. Calculations including facilitator are marked +F; without facilitator are marked –F.

143

Table 21

Number of Edges (Links) and Network Density of Build-ons With and Without Facilitator, Group

2


Facilitator

inclusion

Number of

nodes

Number of

links/edges

Network

density

Number of

links/edges

Network

density


+F 10 23 0.256 80 0.875

–F 9 11 0.153 63 0.700


+F 10 28 0.300 81 0.900

–F 9 15 0.194 64 0.889

Mary’s Misery

+F 10 15 0.156 71 0.789

–F 9 5 0.069 55 0.764


+F 10 10 0.100 37 0.411

–F 9 3 0.280 15 0.194


+F 10 11 0.100 28 0.311

–F 9 5 0.056 18 0.250

Total # of Edges

+F 87 297

–F 39 215

Mean Network Density (SD)

+F 0.182 (0.09) 0.657 (0.28)

Note. Calculations including facilitator are marked +F; those without facilitator are marked –F.

144

Table 22

Summary of Edges/Links and Network Density: Groups 1 and 2

Who Built-On Whose Notes Who Read Whose Notes

Group Facilitator Number

of Nodes

Total Number

of Links/Edges

Mean (SD)

Network Density

Percentage

Total Number

of Links/Edges

Mean (SD)

Network Density

Percentage

1 +F 11 158 0.278 (0.06) 439 0.798 (0.21)

2 +F 10 87 0.182 (0.09) 297 0.657 (0.28)

1 –F 10 75 0.184 (0.59) 349 0.783 (0.22)

2 –F 9 39 0.150 (0.09) 215 0.559 (0.32)

Note. Calculations including facilitator are marked +F, without facilitator are marked –F. Group 1 (n = 11); Group

2 (n = 10)

Comparison between groups (with facilitator) was conducted. Groups 1 mean network density of

build-ons (0.278) was higher than Group 2 (0.182). This is related to the high number of links or

edges created by the Group 1 participants with facilitator. High density measures of notes read

are demonstrated by both groups; however again Group 1 measures (0.798) are higher than

Group 2 (0.657).

Comparison between student groups (without facilitator) was also conducted. Both student-only

groups demonstrated network strength in terms of mean density when the facilitator was

removed. Group 1 build-on network density with facilitator measured 0.278 as opposed to

student only 0.184. Similarly Group 2 build-on network density with facilitator measured 0.182

as opposed to student only, which measured 0.150. Network density of who read whose notes

remained strong.

6.5.1.1 Network density results of t-tests and effect. Results of t-tests on density and effect

size across all five modules with and without Facilitator demonstrate no significant difference

and no strong effects (Table 23). Therefore, it can be concluded that density of student notes

built-ons and notes read were relatively strong in both Groups 1 and 2.

145

Table 23

Comparison of Density with and Without Facilitator of Notes Built-On and Read

Group t df p Cohen’s d

Density (built-on)† 1 .501 9 .628 0.29

Density (read) 1 .160 19 .875 0.07

Density (built-on) 2 .774 17 .450 0.11

Density (read) 2 .712 17 .486 0.33

Note. Group 1 (n = 11); Group 2 (n = 10).

†Probability is corrected for unequal group variances.

In summary, social network densities in both groups were maintained with and without the

facilitator demonstrating power of the student network and active engagement in both building-

on each other’s ideas and/or reading each other’s notes. Social network change across modules

within groups, exemplifies the dynamic flow of sustained work within networks across time.

6.5.2 Social Network: Eigenvector, In-degree, and Out-degree Centrality Results

Results of Eigenvector centrality, in-degree, and out-degree centrality, with and without

facilitator, across all modules, for Group 1 are presented in Table 24 and for Group 2 are

presented in Table 25.

Mean Eigenvector centrality measures of build-ons and notes read are low in both Groups 1 and

2. These low measures are indicative of well-distributed groups. Eigenvector analysis of groups

with and without facilitators demonstrates little change and is therefore indicative of strong

student networks. In-degree/out-degree measures of Group 1 and Group 2 build-ons dropped

markedly in some modules when the facilitator was removed, indicative of a high degree of

facilitator participation.

146

Table 24

Social Network Centrality Measures (in Percentages) With and Without Facilitator, Group 1


Facilitator

inclusion

Eigenvector

centrality (SD)

In-degree

centrality

Out-degree

centrality

Eigenvector

centrality (SD)

In-degree

centrality

Out-degree

centrality

Mr. Singh’s

Pain, Part 1

+F 23.0 (19.5) 61.0 50.0 27.5 (12.3) 17.0 6.0

–F 23.8 (20.8) 35.8 23.5 29.2 (12.0) 19.8 7.4

Mr. Singh’s

Pain, Part 2

+F 24.6 (17.5) 29.0 62.0 28.7 (9.4) 0.0 0.0

–F 27.7 (15.3) 34.6 59.3 30.5 (8.2) 0.0 0.0

Mary’s

Misery

+F 25.3 (16.4) 37.0 37.0 28.9 (8.7) 2.0 2.0

–F 26.9 (16.7) 40.7 40.7 30.5 (8.5) 2.5 2.5

Judy’s Last

Days, part 1

+F 21.9 (20.8) 34.0 56.0 26.2 (14.8) 34.0 23.0

–F 21.2 (23.4) 25.9 13.6 27.9 (15) 25.9 25.9

Judy’s Last

Days, Part 2

+F 23.6 (18.7) 49.0 49.0 26.6 (14.1) 25.0 25.0

–F 24.1 (20.5) 19.8 44.4 28.1 (14.5) 27.2 27.2

Mean (SD)

+F

23.7

(1.3)

42.0

(12.9)

50.8

(9.3)

27.6

(1.2)

15.6

(14.6)

11.2

(11.9)

–F

24.7

(2.61)

31.4

(8.4)

36.3

(18.0)

29.2

(1.3)

16.2

(14.8)

12.6

(13.0)

Note. In-degree centrality (receiving links) and out-degree centrality (linking to others); all centrality measures are

interpreted as the higher the measure the more centralized the network, the less distributed. Calculations including

facilitator are marked +F, without facilitator –F. Group 1 (n = 11); Group 2 (n = 10).

147

Table 25

Social Network Centrality Measures (in Percentages) With and Without Facilitator, Group 2


Facilitator

inclusion

Eigenvector

centrality (SD)

In-degree

centrality

Out-degree

centrality

Eigenvector

centrality (SD)

In-degree

centrality

Out-degree

centrality


+F 23.1 (21.6) 33.3 58.0 29.6 (11.2) 12.4 12.4

–F 23.8 (23.3) 25 25 31.8 (10.1) 14.1 14.1


+F 24.6 (17.5) 29.0 62.0 29.6 (11.3) 11.1 11.1

–F 27.7 (15.3) 34.6 59.3 32.7 (6.5) 12.5 12.5

Mary’s Misery

+F 23.1 (16.4) 44.4 32.2 29.9 (10.4) 23.5 11.1

–F 24.8 (22.2) 20.3 20.3 32.4 (7.8) 26.6 12.5

Judy’s Last Days, part 1

+F 21.9 (20.8) 34.0 56.0 26.7 (17.0) 65.4 16.1

–F 18.4 (27.8) 25.0 10.9 26.9 (19.7) 34.4 20.3


+F 19.4 (25.0) 25.9 25.9 27.2 (16.2) 51.9 14.8

–F 19.0 (27.4) 21.9 21.9 28.6 (17.2) 56.3 14.1

Mean (SD)

+F

22.4

(1.9)

33.3

(7.0)

46.8

(16.5)

28.6

(1.5)

32.8

(24.5)

13.1

(2.2)

–F

22.7

(4.0)

25.4

(5.5) 27.5

(18.5)

30.5

(2.6)

28.8

(17.8)

14.7

(3.2)

Note. In-degree centrality (receiving links) and Out-degree centrality (linking to others); all centrality measures are

interpreted as the higher the measure the more centralized the network, the less distributed. Calculations including

facilitator are marked +F, without facilitator –F. Group 1 (n = 11); Group 2 (n = 10).

6.5.2.1 t-test and effect size results of social network centrality. Significant difference and

large effect size were found for Group 1 when tested with and without Facilitator, on social

network centrality dimensions of in-degree and out-degree of notes built-on (Table 26).

Therefore we can conclude the Facilitator in Group 1 was highly participative in terms of

receiving and linking or building-on others notes and had a large effect.

148

Table 26

Comparisons of SN Centrality Measures With and Without Facilitator, by Group, Across All

Modules

Group t df

a p Cohen’s d

Eigenvector (B) † 1 1.16 13 .269 0.51

In-degree (B)* 1 2.21 19 .039 0.98

Out-degree (B) †* 1 2.29 13 .039 1.01

Eigenvector (R)** 1 3.09 19 .006 1.35

In-degree (R) 1 .09 19 .933 0.04

Out-degree (R) 1 .26 19 .801 0.11

Eigenvector (B) † 2 .22 11 .830 0.10

In-degree (B)* 2 2.73 17 .014 1.26

Out-degree (B)* 2 2.41 17 .028 1.10

Eigenvector (R) 2 1.96 17 .067 0.89

In-degree (R) 2 .41 17 .686 0.19

Out-degree (R) 2 -1.26 17 .223 0.57

Note. B = Built-on notes; R = notes Read. Group 1 (n = 11); Group 2 (n = 10). Results preceded by a negative signs

indicate that the Group 2 mean exceeded the Group 1 mean. a Because data were available only in aggregated form, grouped t-tests were used.

†Probability is corrected for unequal group variances. * p < .05, ** p < .01.

Significant difference and a very large effect size (1.35) was also found for Eigenvector notes

read, with and without Facilitator for Group 1. Therefore the Facilitator was also a strong

participant in reading group notes. Although there was not a significant difference in Eigenvector

notes read, with and without Facilitator for Group 2, the effect size was large (0.89). Therefore it

can be said that the Facilitator of Group 2 was a very active reader.

Like Group 1, Group 2 demonstrated significant difference and large effect sizes on differences

with and without Facilitator for dimensions of in-degree and out-degree, of notes built-on, across

all modules. Therefore the Facilitator in this group could also be considered a very active

member.

149

6.5.2.2 t-test and effect size results of social network density and centrality measures

across groups. Comparison results between Groups 1 and 2, with a facilitator, across all five

modules indicates that density of notes built-on is significantly different (Table 27). Difference

in density for notes built-on was significantly stronger in Group 1 (p < .01) and effect size (1.31).

Therefore we can conclude that substantially more notes were built-on in Group 1 and student

and facilitator participation was stronger and significantly different from Group 2. Density of

build-on notes appears to be the major factor emerging from social network analyses in this case

study that is related to higher levels of knowledge improvement in Group 1, across all modules.

Table 27

Comparisons of SN Measures, between Groups 1 and 2, With Facilitator, Across All Modules

t df p d

Density (Build-ons)** 2.90 19 .009 1.31

Density (Read) 1.31 19 .205 0.57

Eigenvector (B) 1.75 19 .096 0.76

In-degree (B) † 1.93 16 .072 0.83

Out-degree (B) † 0.67 14 .514 0.30

Eigenvector (R) -1.71 19 .104 -0.74

In-degree (R) † -1.93 14 .073 -0.85

Out-degree (R) † -0.52 11 .616 -0.22

Note. B = Built-on, R = Read. Group 1 (n = 11); Group 2 (n = 10). Results preceded by a negative signs indicate that

the Group 2 mean exceeded the Group 1 mean.

†Probability is corrected for unequal group variances. * p < .05, ** p < .01.

All other social network dimensions, of notes build-on and notes read, do not demonstrate

significant difference or effect size between groups. In other words, Groups 1 and 2 are not

significantly different in terms of various centrality measures, i.e. Eigenvector centrality,

receiving notes and linking of notes, both in terms of notes built-on and notes read. Therefore we

can conclude both Groups 1 and 2 were well-distributed, non-centralized networks.

150

6.6 Social Network Analyses Results of Cliques (K)

and Cohesion Index of Build-On Ideas Only

Since social network results of who read whose notes was equally strong across both groups,

further structural analyses of cliques and cohesion measures will focus on detailed examination

of building-on ideas only. In addition, analyses will now be performed only with inclusion of the

facilitator to determine the relationship between facilitator and student interactions.

Results of clique members and cohesion index by module and across groups can be found in

Tables 28 and 29. Although Group 1 and Group 2 clique sizes were similarly composed of three

to four participants, there are marked distinctions between the structures of cliques across

groups. Group 1 had almost twice as many total cliques (27) as Group 2 (14).

Most striking is the fact that the Group 1 facilitator participated in all 27 cliques, which is more

than three times as many as the Group 2 facilitator, who participated in eight of 14 cliques.

Despite the very high level of Group 1 facilitator engagement the Cohesion Index across

modules was lower than Group 2, indicative of a more highly distributed and inclusive social

networks. Interclique connectivity was maintained in both groups by the numerous participants

that were active across many cliques. “This ‘bridging’ phenomenon provides for a wealth of

information flowing to all members” (Aviv et al., 2003, p. 13).

Important findings emerged from this analysis. Findings of high-level facilitator participation

within well-distributed network provokes new questions, particularly in regards to scaffolding

and supporting enhanced knowledge improvement, as was found in Group 1.

151

Table 28

Clique Members and Cohesion Index Results of Build-Ons by Group With Facilitator

Group 1 Group 2

Number of

Cliques (K) in

each Module

Members Clique

Cohesion Index

Members Clique

Cohesion Index

Mr. Singh’s

Pain, Part 1

K1 11,6,7,9,10,5 3 2,6,9,1 3

K2 11,6,7,9,10,1 3.333 2,6,9,4 3

K3 11,6,3 2 2,6,10 2.625

K4 11,2,5 2.182 2,5,3 3

Mr. Singh’s

Pain, Part 2

K1 9,11,7,5,10 2.143 10,3,5,7,1 2.778

K2 9,11,7,6 1.647 10,3,5,2 2

K3 9,11,7,8 1.867 10,3,5,8 1.846

K4 9,11,3,6 2 10,3,9,8 2.4

K5 9,11,1,5 2

K6 9,11,2,6 1.867

K7 9,11,2,10 1.867

Mary’s Misery

K1 9,11,10,6 2.333 3,10,8,1 4.8

K2 9,11,10,7 2.154 3,10,5 3.5

K3 9,11,10,5 2 9,1,10 3.5

K4 9,11,1 2

K5 9,11,3,5 2.333

K6 8,11,7 2.667

K7 8,11,5 2.4

Judy’s Last

Days, Part 1

K1 11,7,8,9 4.667 10,3,2 7

K2 11,7,1 3 10,3,5 7

K3 11,3,2 4.8

K4 11,6,9 3

K5 11,6,1 3.429

Judy’s Last

Days, Part 2

K1 6,11,7,8,9 3.333 1,3,5,10 n/a

K2 6,11,7,1 2.8

K3 6,11,3,8 2.8

K4 6,11,2,9 2.8

M (SD) 2.70 (0.71) 4.26 (1.93)

Note. Group 1 n = 11; Group 2 n = 10. K = Clique. Facilitators = #11 (Gr. 1) and #10 (Gr. 2).

152

Table 29

Clique and Cohesion Index Results of Build-On Notes in Groups1 and 2, With Facilitators

Who built-on whose notes

2008/2009

Group

Total number

of cliques

Average size

of cliques %

(SD)

Mean clique

cohesion index

% (SD)

No. cliques

facilitator

belongs to


1 4 4.50 (1.73) 2.63 (0.64) 4

2 4 3.50 (0.58) 2.91 (0.19) 1


1 7 4.14 (0.38) 1.91 (0.16) 7

2 4 3.50 (0.58) 2.91 (0.19) 1

Mary’s Misery

1 7 3.57 (0.53) 2.27 (0.24) 7

2 3 3 4.2 3

Subtotal # and Mean (SD)

1 18 4.07 (0.47) 2.27 (0.36) 18

2 11 3.33 (0.29) 3.34 (0.74) 5


1 5 3.2 (0.45) 3.78 (0.89) 5

2 2 3 7 2


1 4 4.25 (0.5) 2.93 (0.27) 4

2 1 4 n/a 1

Total # and total mean (SD)

1 27 3.932 (0.53) 2.70 (0.71) 27

2 14 3.4 (0.42) 4.26 (1.93) 8

Note. Higher Cohesion Index = more isolated cliques. Group 1 n = 10; Group 2 n = 11.

153

6.7 Relationship of Social Networks to Knowledge Improvement Scores

6.7.1 Relationship of Social Network Structural Analyses of the Three Pain Modules to

Pain Knowledge Improvement Pre-/Posttest Scores

The relationship of various dimensions of social network structural analyses were examined

alongside the results of notes built-on in three pain modules, for comparison between groups, and

with pain knowledge pre-/posttest results. Tests for significant difference and effect size were

also conducted to determine if there is a relationship between social network participation

patterns and knowledge improvement.

Results of clique and Cohesion Index (Table 30) for the three pain modules indicate that the

Group 1 facilitator participated in all 18 cliques for the three pain modules. While the Group 2

facilitator only participated in less than 50% (5/11) of the cliques. Mean clique Cohesion Index

was lower in Group 1 (2.27) than Group 2 (3.33), indicative of the fact that the group on the

whole was more distributed or democratized. Group 1 participants created approximately 40%

more links than Group 2 and correspondingly network density measures were higher in Group 1

(0.315) than Group 2 (0.237) (Table 31). Eigenvector and out-degree centrality measure are

similar between groups. In-degree centrality for Group 1 is 42.33, as compared to 27.35 for

Group 2.

Relationship patterns indicate that higher performance on pre-/post knowledge tests in Group 1

were related to higher levels of build-on notes in KF, higher number of network edges/density,

lower coherence index and centrality measures, as well as a greater number of cliques, and more

facilitator engagement in an evenly distributed network.

Group 1 network density of build-on notes demonstrated significant difference compared to

Group 2 (p <.01), as well as an extremely strong effect size of 1.40 (Table 32). Lower cohesion

of Group 1 build-ons was also statistically significant (p < .001) when compared to Group 2,

with an effect size of 1.84. In-degree centrality, receiving of links, was also significantly stronger

in Group 1 than Group 2 (p<.05), with an effect size of 1.18. Eigenvector and out-degree

centrality measures were not significantly different indicating that both groups were well-

distributed and many links between participant notes and ideas were made.

154

Table 30

Comparative Summary of SN Results of Groups 1 and 2 in Three Pain Modules

Cohesion

index % (SD)

No. of

edges/

links

Density Eigenvector centrality

%

In-degree centrality

%

Out-degree

centrality %


G1 2.63 (0.64) 38 0.345 23.0 61.0 50.0

G2 2.91 (0.19) 23 0.256 23.1 33.3 58.0


G1 1.91 (0.16) 38 0.336 24.6 29.0 62.0

G2 2.91 (0.19) 28 0.300 27.0 28.4 65.4

Mary’s Misery

G1 2.27 (0.24) 30 0.264 25.3 37.0 37.0

G2 4.2 15 0.156 24.8 20.3 20.3

Total

G1 106

G2 66

Mean (SD)

G1

2.27

(0.36)

0.315

(0.04)

24.3

(1.1)

42.3

(16.6)

49.7

(12.5)

G2 3.34

(0.74)

0.237

(0.07)

25.1

(2.0)

27.4

(6.6)

47.9

(24.2)

Note. G = Group; Group 1 n = 11, Group 2 n = 10.

Table 31

Comparisons of Social Network Measures of Build-Ons, Between Groups, Across Three Pain

Modules

t df p Cohen’s d

Cohesion Index † *** -4.15 13 .001 -1.84

Density (B) † ** 3.09 14 .008 1.40

Eigenvector (B) -1.10 19 .284 -0.48

In-degree (B) † * 2.76 13 .016 1.18

Out-degree (B)† .22 13 .841 0.09

Note. Group 1 n = 11, Group 2 n = 10. Numbers of cases differ among analyses due to uneven group participation.

Negative (-) sign before number = stronger in Group 2 than Group 1. B = Build-on.

†Probability is corrected for unequal group variances

* p < .05, ** p < .01, *** p < .001.

155

Table 32

Results of 2008/2009 Pain Pre-/Posttests

Pre Post

Groupa M (SD) 95% CI

M (SD) 95% CI Difference d

b

1 0.68 (0.14) [0.59, 0.77] 0.84 (0.12) [0.59, 0.76] 0.16 1.11

2 0.66 (0.11) [0.76, 0.92] 0.78 (0.09) [0.71, 0.86] 0.12 1.12

Note. The difference between the pre- and posttest means combined over group was significant, F (1,15) = 17.94, p

< .001. a Group 1 n = 8; Group 2 n = 9.

b Cohen’s measure of effect size.

6.7.2 Correlation Between Results of Social Network Analyses and Pain Posttest Scores

The data for Mr. Singh Parts 1 and 2 and Mary’s Misery were reformatted and read into SPSS

for Group 1. The posttest score was correlated to the Eigenvector score, the in-degree and out-

degree measures. Because of the low number of cases nonparametric correlation (Spearman’s

rho) was used. Over the three scenarios, only the in-degree measure was correlated significantly

for Mr. Singh part 1 only to posttest (r = .724, p = .042) . The remaining correlations to post (or

difference) scores were not significant at = or < .1.

For Group 2 the results with retention of zero network scores were not significant for Mr.

Singh’s Pain, Part 1. However, when student number 2 (with zero activity) was omitted, the

picture changed. Posttest was related significantly to Eigenvector (rho = .822, p = .012), to in 1

(rho = .740, p = .036), and to out-degree centrality (rho = .682, p = .063). The results for Group 2

on Mr. Singh part 2 or Mary’s Misery were not significant either with or without the cases with

zero activity.

Groups 1 and 2 were combined for further analysis. Table 33 shows nonparametric correlations

along with significance of the posttest score with the social network variables. For this analysis

the zero measures were retained. Of the nine correlations, all are positive and all are above .2, six

above .3. Two are significant at p < .05 (Eigenvector centrality in Mr. Singh’s Pain, Part 1 and

in-degree in Mary’s Misery) and 2 at p < .1 (in-degree in Mr. Singh’s Pain, Part 1 and out-degree

in Mary’s Misery).

156

Table 33

Spearman Correlations of the Post Score with Social Network Variables

Variable Eigen

Mr S 1

In

Mr S 1

Out

Mr S 1

Eigen

Mr S 2

In

Mr S 2

Out

Mr S 2

Eigen\

MM

In

MM

Out

MM

Correlation .461 .516* .262 .370 .220 .257 .399 .429 .511

*

Significance .063† .034* .309 .144 .395 .319 .112 .085† .036*

Note. * p < .05, † p <.1.

In conclusion, this analysis demonstrates important and strong relationships between Group 1

knowledge improvement pre-/posttest outcomes and social network measures of high build-on

note density, low cohesion index, low Eigenvector centrality, accompanied by significant

measures of centrality, confirming key Knowledge Building principles of the importance of

distributed networks, decentralized participation, and democratization of ideas in a communal

space.

6.8 Cluster 2 Summary of Performance Measures (Over and Above Traditional

Learning) From Social Network Structural Results

Cluster 2 examined social network structural measures, over and above and above traditional

learning measures to determine relationships between Knowledge Forum activity/interactivity,

network density, centrality, cohesion, clique measures, and their relationship to pre-/posttest

outcomes.

This cluster examined three questions:

What are the participant online activity and interactivity measures, by module, by group, and by

year, in the 2005–2009 study? The Knowledge Forum analytic toolkit was used to determine

read, write, build-on measures. Participation was high across all categories. Notes read were

particularly high. This measure may or may not be reliable. It is not known if participants read or

merely opened/closed notes without reading them. Build-ons notes are seen in this study, as the

most reliable indicator Knowledge Building.

The second question in this cluster was: Is there a significant difference in 2008/2009 Groups 1

and 2 pain knowledge scores from pre- to posttest and is there a significant difference between

157

groups? Result indicated pain knowledge improved significantly in both groups; however,

knowledge increase was greater in Group 1 than in Group 2.

Therefore, the third question asked was: What are the social network structural differences

between groups that supported increased knowledge improvement and how are these differences

related to centralization and/or democratization of participation and ideas?

Consistent with the findings of greater knowledge gains in Group 1 than in Group 2, are patterns

of higher-level build-on activity, number of edges, and network density. Social network analysis

was used to evaluate distribution of participation and ideas in each group. These analyses

demonstrated similar Eigenvector centrality of well-distributed networks in both groups, in

regards to ‘who built-on whose notes’ and ‘who read whose notes’. Mean effect of ‘statistical’

removal of facilitator in both groups was negligible, attesting to the fact that both Group 1 and

Group 2 represented structurally strong and well-distributed student networks. Cohesion Index of

differed. Mean Cohesion Index of Group 1 was substantially lower than Group 2, noting

differences between groups, across cliques, within modules.

Most remarkable is the number of cliques created by Group 1 participants, as well as the overall

number of cliques the facilitator belonged and the strong interclique participation by students.

Contrary to popular notions of democratization in education, the expert/teacher/facilitator may be

extremely active as a “partner” and co-creator of knowledge within a social network.

Atypical high levels of facilitator participation require further examination in relationship to

students within the network. This will be examined in this study using social network analysis

measures and maps for analysis of network position. Social network position structures are

related to power structures. Results of these analyses can be interpreted according to centrality of

position and power of ideas. Results of the next social network analysis will begin with

identification of who is at- the-”structural” core (facilitator and/or students), mid and periphery

and track power dynamic/flows/shifts across modules (Aviv et. al., 2003).

Results presented herein, demonstrate structural relationships are indicative of high functioning

group, where the facilitator, plays a key role in knowledge work with students. These social

network structural relationships are also indicative of Knowledge Building symmetry, collective

responsibility, and epistemic agency important to the democratization of continuing medical

education.

158

At this point, we can conclude that there is a strong relationship between increased knowledge

gains, high levels of participant online activity, and communal work that includes knowledge

experts among participants in decentralized, opportunistic environments. These relationships

may be highly integrated with notions of position and power in the social network,

democratization of ideas and Knowledge Building discourse.

As shown in this chapter, structural social network analysis helped us to explain dynamic

interactions and relationships important to the improvement of network distribution and

democratization Knowledge Building in continuing medical education. These types of social

network analyses enables us to consider how new cultural tools, such as Knowledge Forum,

enable new participant structures and interactions that have the potential to change asymmetries

and traditional pedagogic paradigms.

The current analysis also evoked further questions particularly about facilitator/student

interactions and relationships. Both facilitators were highly active in the social networks. What

we need to know is: How did they participate? Did they didactically dominate the group or

facilitate and scaffold knowledge building? In social network analysis terms, did they share the

centre space and create opportunities for democratized engagement or was the environment

autocratic, represented only by the facilitator ideas-at-the-centre? We need to know what were

the sociocognitive dynamics that enabled knowledge work over and above learning?. Results of

social network position/power analysis, as well as content analysis of the discourse will be

presented in the next chapter to help determine answers to these and other questions.

159

CHAPTER 7


SOCIAL NETWORK POSITION/POWER ANALYSIS AND SOCIOCOGNITIVE

DYNAMICS OF KNOWLEDGE BUILDING

7.1 Introduction

Chapter 7 presents results based on the third cluster of subquestions, regarding the sociocognitive

dynamics that enable work over and above traditional learning. Cluster 3 is guided by the

following research subquestion: What are the social network relationships between structural

position, power (defined as centrality of ideas), and knowledge improvement; and how are these

relationships reflected in facilitator/student sociocognitive dynamics of Knowledge Building

(beyond learning objectives), through emergent themes, in complexity of discourse, and by

indicators of Knowledge Building? Table 34 summarizes data analyses and data sources used to

determine the results herein.

Concepts of power, dually defined as structural position and centrality of ideas are examined in

the first analysis in this chapter. The centrality or power of one’s ideas are explored in relation to

knowledge improvement. Centrality maps and corresponding measures of position/power of

facilitator/students are provided across all 10 modules; results are compared across groups.

Content analysis of discourse notes was conducted on forty percent of the dataset; two modules

from each of the two groups was selected for within note analyses. Results of four analyses are

presented, including: (a) facilitator/student sociocognitive dynamics, (b) identification of

emergent themes beyond those predetermined by learning objectives, (c) complexity of clinical

discourse, and (d) Knowledge Building indicators and exemplars. Results of within note analyses

aim to further our understanding of the sociocognitive dynamics that support adductive and

abductive Knowledge Building, working with ideas more deeply and more broadly.

160

Table 34

Cluster 3: Social Network Analysis of Sociocognitive Dynamics



Subquestion

6





Data Analyses Data Sources


(measures and mapped visualizations)

Structural analysis of position/idea centrality/power

and relationship to knowledge improvement


build-ons (with facilitator),

= 10 social network centrality maps

Identification of whose ideas are at the core, mid,

or periphery (facilitator and/or students; Aviv

et al., 2003)

Relationship of student network position to

individual pre-/posttest knowledge gain Pre-/posttest results by individual and

social network centrality maps


Facilitator stance

Facilitator as monitor, mentor, participant, or

expert (modified protocol by Tabak

&Baumgartner, 2004)



20% by second rater), within-note analysis


work?

As above







(40% of dataset) to be scored by PI,

within-note analysis

Relationship of emergent themes to abductive and

adductive knowledge improvement

As above



(Bordage, 1994), 4-pt rating scale of clinical

discourse as reduced, dispersed, elaborated, or

compiled



30% by second rater)


Evidence of Knowledge Building principles,

particularly community, democracy, agency, and

improvable ideas related to Knowledge Building

intentionality and emergent ideas/themes



second rater)

Use of scoring protocol by Sibbald (2007) and

definitions by Scardamalia (2002)


161

7.2 Results of Social Network Position/Power Analysis

Ideas of power relationships frame investigations in this next section (Foucault, 2001). The

notion of position within a social network has two sides. It may refer to the structural position an

individual may hold within the social network relative to others. It may also refer to the power or

centrality of knowledge and ideas; this interpretation can be thought of as semantic in nature and

related to content analysis, within note analysis of discourse results in the next section.

Results of social network structural position/power analyses presented in this section include:

1. Relationship of student network position to individual pre-/posttest knowledge gain;

2. Social network structural analysis results of position/power represented in centrality

maps and corresponding centrality measures.

7.2.1 Relationship of Individual Student Social Network Position Scores to Difference on

Pre-/Posttest Knowledge Scores

Position/power and knowledge improvement relationships were identified through examination

of the 2008/2009 Group 1 (Table 35) and Group 2 (Table 36) individual pre-/posttest scores and

centrality position measures.

Table 35

Group 1 Individual Differences in Student Pain Knowledge Pre-/Posttest Scores

Group 1 individual pre- and posttest scores 2008/2009

Student Pretest score Posttest score Difference

Albert 0.58 0.90 0.32

Craig 0.87 0.87 0.00

Don 0.56 0.59 0.03

Francis 0.63 n/a n/a

Gail 0.59 0.95 0.36

John 0.69 0.73 0.04

Jeff 0.83 n/a n/a

Len 0.80 0.88 0.08

Mary 0.85 0.85 0.00

Sara 0.51 0.93 0.41

162

Table 36

Group 2 Individual Differences in Student Pain Knowledge Pre-/Posttest Scores

Group 2 individual pre- and posttest scores 2008/2009

Student Pretest score Posttest score Difference

Brenda 0.57 0.85 0.28

Jack 0.66 0.83 0.16

Jenn 0.80 0.85 0.04

Jeorge 0.61 0.78 0.16

Maria 0.64 0.78 0.14

Phil 0.49 0.57 0.09

Rose 0.65 0.76 0.11

Shira 0.78 0.88 0.10

Sven 0.78 0.78 0.00

Individual pre-/posttest scores were calculated in SPSS for each student in Groups 1 and 2.

Students were categorized as greatest gainers, incoming stars, or others. All were included in the

position/power analyses. However, greatest focus was given to greatest gainers, to see what

positions they held in module and if, and how these positions changed across modules over time.

Incoming stars or those individuals that participated in the course with an incoming high level of

knowledge were particularly noteworthy; it was hypothesized that these individuals would take

on central roles, like or near the facilitator.

Three students in Group 1 and 1 student in Group 2 were identified as greatest gainers, those that

made the largest gain from pre- to posttest score. In Group 1 they were Albert (32% gain); Gail

(36% gain); and Sara (41% gain).

In Group 2 the one student was Brenda (28% gain).

Three students in Group 1 and one student in Group 2 were identified as incoming stars; their

pretests scores were 80% or higher and their posttest scores were 85% or higher. In Group 1 they

were Craig, Len, and Mary.

In Group 2 the one student was Jenn.

7.2.2 Social Network Position, and Power Maps and Measures

Social network position/power analyses were done across 2008/2009 Groups 1 and 2, on all five

online modules in each group, for a total of 10 modules. Only build-on notes were analyzed.

163

Early social network centrality analyses showed build-on notes were more variable across

groups, while notes read were very homogeneous—all at a high level. Therefore build-on notes

were chosen to examine possible differences.

The Group 1 facilitator is number 1 and the Group 2 facilitator is number 2. Facilitators are

marked by a blue square and students are marked by red dots on the centrality maps. As in other

analyses, all real names were replaced with fictitious ones. Nonparticipants in the module are

indicated in tables, but are not removed from the centrality maps.

Netminer 3 software was used to perform social network position analysis to identify who is at

the structural centre, as well as who is in midfield and who is at the periphery (Aviv et al., 2003).

“Who” is identified as “facilitator” or “student.” Five modules for each of the two groups were

mapped. Social network centrality measures were also calculated to enable interpretation of the

relationships mathematically as well as visually. Dynamic shifts in position over time, across

modules was noted. In particular, the shift of three Group 1 students to the centre and a fourth

near the threshold, dramatically displays the transition of power, epistemic agency, and the

democratization ideas in the problem space. Interesting relationships between social network

position and greatest knowledge gain were found. The results of position/power analyses, related

centrality scores, and pain pre- to posttest improvement scores are presented below for each of

the five modules in each group.

7.2.2.1 Position/power results for Group 1, Mr. Singh’s Pain, Part 1. Results indicate that

the central positions were occupied by the facilitator and 2 students, Gail (greatest gainer) and

Mary (incoming star; Figure 61). In terms of Eigenvector centrality the facilitator held the most

power. However, Eignevector, in-degree, and out-degree measures for the “greater gainer” and

“star” are very similar. This level of epistemic agency from student #5 is particularly

noteoworthy in the firt module. Ideas in the problem space were well distributed, with two

students in the centre, three midfield, and four on the periphery (see Table 37).

164

Table 37

Social Network Position/Power Map and Centrality Measures for Group 1, Mr. Singh’s Pain,

Part 1, Build-On Notes

Mr. Singh’s Pain Part 1, build-on notes

Group 1 Participant Eigenvector Centrality map position

Albert 0.107 periphery

Craig 0.06 periphery

Don 0.046 periphery

Francis 0 n/p

Gail 0.444 core

John 0.199 mid

Jeff 0.217 mid

Len 0.034 periphery

Mary 0.446 core

Sara 0.387 mid

Facilitator 1 0.591 core

Distribution 3 3 4 + 1 n/p

Note. n/p = nonparticipant.

165

Figure 61. Social network position/power map Group 1, Mr. Singh, Part 1, Build-on notes.

7.2.2.2 Position/power results for Group 1, Mr. Singh’s Pain, Part 2. The central position is

shared by facilitator and 1 student, Mary (Figure 62). This student had the highest pretest score

of the group (85%) and therefore it is understandable why she would adopt a central shared

position in the social network and potentially adopt a similar role to the faciltator. The majority

of students are positioned in the peripheral field. Gail have moved from the centre to the

peripheral field, while Sara has moved to midfield (see Table 38).

166

Table 38




Group 1 Participant Eigenvector score Centrality map position

Albert 0.205

periphery


Don 0.087 periphery

Francis 0 n/p

Gail 0.196 periphery

John 0.175 periphery

Jeff 0.176 periphery

Len 0.33 mid

Mary 0.493 core

Sara

0.308 mid




167

Figure 62. Social network position/power map Group 1, Mr. Singh, Part 2, Build-on notes.

7.2.2.3 Position/power results for Group 1, Mary’s Misery. Central position remains shared

by facilitator and the same student, Mary (Figure 63). The majority of students have shifted

position now to the midfield. Albert and Gail have now shifted position from the periphery to

midfield, while Sara remains at midfield (see Table 39).

168

Table 39

Social Network Position/Power Map and Centrality Measures for Group 1, Mary’s Misery,

Build-On Notes

Mary’s Misery, build-on notes


Albert 0.223

mid

Craig 0 n/p

Don 0.111 periphery

Francis 0.083 periphery

Gail 0.239 mid

John 0.157 periphery

Jeff 0.26 mid

Len 0.362 mid

Mary 0.483 core

Sara 0.287

mid




169

Figure 63. Social network position/power map Group 1, Mary’s Misery, Build-on notes.

7.2.2.4 Position/power results for Group 1, Judy’s Last Days, Part 1. Although the central

field remains shared by facilitator and a student, it is now a different student, Albert, who is one

of the highest achievers (90%), as well as one of the largest knowledge gainers (Figure 64).

Mary, formerly at the structural centre has now shifted position to midfield. Students are well

distributed across mid and peripheral fields. “Greatest gainer,” Gail and Sara have not only

shifted position from midfield to the periphery, but with zero scores have not built-on notes (only

read as indicated in the ATK activity analysis). See Table 40.

170

Table 40

Social Network Position/Power Map and Centrality Measures for Group 1, Judy’s Last Days,


Judy’s Last Days Part 1, build-on notes


Albert 0.5

core


Don 0.062 periphery

Francis 0 n/p

Gail 0 n/p

John 0.25 mid

Jeff 0.275 mid

Len 0.23 mid

Mary 0.399 mid

Sara 0

n/p




171

Figure 64. Social network position/power map Group 1, Judy’s Last Days, Part 1, Build-ons.

7.2.2.5 Position/power results for Group 1, Judy’s Last Days, Part 2. In this last module,

the central field becomes prime real estate (Figure 65). It is shared to an amazing degree by the

facilitator with three students, one of which is Mary, one of the three “incoming stars.” The other

two are new to this central position, Jeff and John. The greatest “gainers” in the group, Albert,

Gail and Sara shifted position from the centre—to the periphery where they have not created any

build-on notes.

It is worth noting that although three students that came into the course with high pretest scores

only one came close to matching the position/power of the facilitator, Mary. The other two did

not. Craig paricipated on the periphery throughout and Len participated in the midfield

throughout the course. It is not known what caused these participants to hold these positions.

172

Half of the participants now occupy the centre in this last module. This can be said to be a shift

toward democracy, where 3 student share the power of ideas and prestige of centrality with the

facilitator. Interestingly the facilitator has retained a strong central position in the problem space.

However, the facilitator shared the central problem space with one or more students in all five

modules (see Table 41).

Table 41





Albert 0.142

periphery

Craig 0.204 mid

Don 0.14 periphery

Francis 0 n/p

Gail 0 n/p

John 0.413 core

Jeff 0.375 core

Len 0.361 mid

Mary 0.406 core

Sara 0

n/p



Note. n/p = nonparticipant

173


174

7.2.2.6 Position/power analysis interpretation and conclusions for Group 1. The greatest

“gainers” in Group 1, Albert, Gail and Sara shifted position from the core to the periphery across

modules. Albert worked in the periphery of the problem space in the first two modules and them

moved to the midfield in the third module and then to the core and then back to the periphery in

the last module. Although Albert created made few note contributions in the last module , he

read almost all notes (85/96 notes in Module 1; 91/101 in Module 2, and 75/78 in Module 3).

Gail began in the core and then moved to the periphery in the second module and then to the

midfield for the third module. Gail read 95/96 notes in Module 1; 93/101 in Module 2; 70/78 in

Module 3. Sara began in the midfield and remained there for the next two modules and then

moved to the periphery. Sara read 93/96 notes in Module 1; 95/101 in Module 2; 74/78 in

Module 3. Gail and Sara did not participate in the last two modules.

Hence we see a complex relationship of position/power characteristics emerging that defines

greatest “gainer”: (a) depends on incoming level of knowledge, (b) participation for greatest

knowledge gain can be from any position within the problem space, core, mid or periphery, and

(c) gains are evidenced not only by build-on note contributions but also through reading and

understanding others ideas in the problem space. It appears that there is much to be gained from

work at the periphery and midfield, contrary to intuitive understanding and other pedagogic

models such situated learning (Lave & Wenger, 1991).

It is also noteworthy that the core of the Group 1 problem space was always shared by the

facilitator with various students. In Module 1 the core was shared by the facilitator with two

students; in Modules 2, 3, and 4 the core was shared by the facilitator with one student; and in

Module 5 the core was shared by the facilitator and three students. Although the facilitator was

very active, the core was open and in fact demonstrated a strong increase in the final module for

shared opportunities of position and power.

It was hypothesized that the incoming “stars” would most often share the core problem space and

in fact this was found to be true, but only in one of three participation patterns. As expected,

Mary shared position and power with the faciliator in four of five modules. However, Len and

Craig did not participate in the core at all. Len participated in the midfield in four of five

modules and in the periphery; while Craig participated in periphery in four of five modules and

175

in the midfield in the final module. Hence is appears that not all “stars” want to actively

contribute or teach/facilitate.

7.2.2.7 Position/power results for Group 2, Mr. Singh’s Pain, Part 1. The core position in

Mr. Singh’s Pain part 1 is occupied by Facilitator 2 and Shira, one of the highest scorers on pain

pretest (Figure 66). Of particular note is Brenda who achieved the greatest knowledge gain

(28%) and one of the highest posttest scores (85%) in Group 2, who in this module holds a

position in the periphery. Although Brenda did not contribute many build-on notes, she has read

almost all notes (66/69). More than half of the participants occupy a position in the periphery of

the problem space in terms of build-on notes. Like Brenda, Jeorge, Shira, and Sven have all read

almost all notes. Only Phil has read less than 50%. One student, Jack, did not create build-on

notes in this module but read all notes (Table 42). Jenn had the highest incoming score in Group

2 and is positioned in midfield in terms of build-ons notes but has read all notes this first module.

Table 42





Brenda 0.202 periphery

Jack 0 n/p

Jenn 0.353 mid

Jeorge 0.07 periphery

Maria 0.413 mid

Phil 0.045 periphery

Rose 0.014 periphery

Shira 0.509 core

Sven 0.074 periphery




176

Figure 66. Social network position/power map Group 2, Mr. Singh’s Pain, Part 1, Build-ons.

7.2.2.8 Position/power results for Group 2, Mr. Singh’s Pain, Part 2. In Mr. Singh’s Pain

part 2, the core position is shared by the Facilitator 2 and 1 student, Jenn, the incoming star

(Figure 67). Like Mary in Group 1, Jenn entered the course with a relatively high level of

palliative care knowledge, scoring 80% on pretest. Brenda, the greatest gainer in this group has

moved from the periphery to a midfield position. All other students are well distributed across

the mid and pheripheral fields. Jeorge did create build-on notes in this module but read 58/60

notes (Table 43).

177

Table 43





Brenda 0.319 mid

Jack 0.164 periphery

Jenn 0.404 core

Jeorge 0 n/p

Maria 0.37 mid

Phil 0.056 periphery

Rose 0.235 mid

Shira 0.268 mid

Sven

0.288 mid




178

Figure 67. Social network position/power map Group 2, Mr. Singh’s Pain, Part 2, Build-ons.

7.2.2.9 Position/power results for Group 2, Mary’s Misery. Central field position is held by

Facilitator 2 and not shared by any students in this module (Figure 68). Jenn has shifted position

from the core to the midfield. Brenda remains in midfield. Others are distributed across midfield

with only one student in the peripheral field. Jack and Jeorge did not create build-on notes in this

module but read all notes. Phil did not create any build-on notes and read less than 50% of the

notes. See Table 41.

179

Table 44

Social Network Position/Power Map and Centrality Measures for Group 2, Mary’s Misery,

Build-On Notes

Mary’s Misery, build-on notes


Brenda 0.404 mid

Jack 0 n/p

Jenn 0.312 mid

Jeorge 0 n/p

Maria 0.356 mid

Phil 0 n/a

Rose 0.058 periphery

Shira 0.215 mid

Sven 0.26 mid


Distribution 1 5 1 +3 n/p


180

Figure 68. Social network position/power map Group 2, Mary’s Misery, Build-on notes.

7.2.2.10 Position/power results for Group 2, Judy’s Last Days, Part 1. The structural core is

shared by Facilitator 2 and Jenn once again. Jenn shifted position from the midfield to the core in

this module (Figure 69). Brenda shifted position from midfield to the periphery. Although

Brenda and Jack are in peripheral positions, Brenda read 35/36 notes and Jack read all. Five of

the 9 students did not create build-on notes in this module. Juan and Phil read less than 10% of

notes; Rose read less than 50%; while Shira read all notes and Sven read 33/36. See Table 45.

181

Table 45





Brenda 0.119 periphery

Jack 0.114 periphery

Jenn 0.631 core

Jeorge 0 n/p

Maria 0.32 mid

Phil 0 n/p

Rose 0 n/p

Shira 0 n/p

Sven 0 n/p




182


7.2.2.11 Position/power results for Group 2, Judy’s Last Days, Part 2. The core position in

Judy’s Last Days part 2 is shared by the Facilitator 2 and a student that has not been at the core

before, Maria (Figure 70). Jenn shifted position from the core to midfield. Brenda shifted

position from the periphery to midfield. The same five students, as in the previous module, did

note create build-on notes in this module. See Table 46.

183

Table 46





Brenda 0.359 mid

Jack 0 n/p

Jenn 0.403 mid

Jeorge 0 n/p

Maria 0.506 core

Phil 0 n/p

Rose 0 n/p

Shira 0 n/p

Sven 0 n/p


Distribution 2 2 6 n/p


184


It is noteworthy that various students have taken on position at the centre. Moveover, the

position at the centre was never occupied by the student making the greatest knowledge gains,

student Brenda. This student participated in the mid and peripheral fields only. Jenn participated

the most often at the structural core of the social network, like Mary in Group 1, Jenn came into

the program with a high level of knowledge as evidenced by her pretest score. Therefore the

pattern appears across groups, indicating that some (but not all) students with high levels of

knowledge in the subject area will share position and power with the facilitator. Overall

positional analysis in both Groups 1 and 2 are well distributed. The dynamic and changing nature

of students positions within a social network over a long period of time should be noted (1 month

per module, across five online case-based modules). The persistent central position of

185

Facilitators 1 and 2 denotes their communal engagement in the Knowledge Building process.

However, this structural analysis of position and power does not inform us as to how the

facilitators engage with participates. Can the facilitator/student structure be characterized to

determine what kind of stance is most productive in scaffolding Knowledge Building, that of

monitor, mentor, and/or partner as described by Tabak and Baumgartner (2004).

7.2.2.12 Position/power analysis interpretation and conclusions for Group 2. Centrality

mapping and power analysis demonstrated facilitator/student shared power spaces in four of the

five Group 2 modules. Facilitator 2 remained at the core in all modules. Students with the

greatest incoming knowledge test scores were the ones that shared the central power space with

facilitators initially. Advances in knowledge gains were demonstrated by those that participated

at the periphery in terms of build-on notes; many of these gains were supported by exemplary

reading scores of notes read. This finding supports Zhang and colleagues’ (2002) study that

focused on the importance of strong reading scores to support Knowledge Building.

In conclusion, students with high levels of incoming knowledge may tend toward sharing the

core problem space and role of facilitation—but not always. Position and power may change

over time. Peripheral positions in a number of cases, in this study, led to greatest knowledge

advances. We are therefore in need of reconceptualizing commonplace associations of position

and power, and our notions of what constitutes participation. Apparently there is much to be

gained from working at the periphery and sometimes this work is through note contributions but

other times it is through high levels of reading of others ideas. Democratization of Knowledge

Building was demonstrated through a stunning shift is position by students to the core in the final

module in Group 1 and continuous sharing of core position and power with participants in

Group1; shared position and power was also evident in Group 2 but not across all modules and

not to the same degree.

Further research is required to understand the complexities of social network power

relationships, beyond structural findings of centrality position, involving how ideas-at-the-centre

of a problem space are worked with, in terms of the power of an idea, participant previous

knowledge, and personal preferences for knowledge work. These sociocognitive relationships are

further expressed in the results of within notes analyses of discourse. Content analyses results

will be presented next.

186

7.3 Results of Social Network Content Analyses

This section presents results of within note content analysis to further advance explanations and

understandings of previous social network structural and position/power results. Content

analyses results herein relates specifically to the details stated in the second part of my 6th

research subquestion: What are the social network structural and content relationships between

position/power and knowledge improvement; and are these relationships reflected in

facilitator/student characteristics, complexity of discourse, themes of belief and design-mode

work, and the demonstration of indicators of Knowledge Building?

Results of the following series of analyses are presented below:

· Relationships between facilitator and students, demonstrated through facilitator

stance and comparison of facilitator/student patterns of statements and questions;

· Thematic results of content: emergent ideas, above and beyond learning objectives

and relationship to adductive and abductive knowledge work;

· Complexity of discourse using Bordage scale for Semantic Analysis of Clinical

Discourse (1991); and,

· Results of analysis of Knowledge Building indicators and exemplars.

Based on the results of the previous section on centrality position/power analysis four modules

were selected, two modules from Group 1 and two modules from Group 2. Mr. Singh’s Pain,

Part 2 was selected based on the fact that these modules in both groups were represented by a

student participant and facilitator at the core and well distributed networks. In addition, this

module was one month into the program, so participants were well acquainted with KF technical

functions and social engagement online. In terms of content, this module deals with some key

principles reflected in items on the pain knowledge and beliefs pre-/posttests.

The second modules selected for further analyses across both groups was the final module,

Judy’s Last Days, Part 2. This module were specifically selected for within note examination to

help provide further understanding of the strong shift in position by numerous Group 1 students

to the core.

187

7.3.1 Social Network Analysis of Facilitator/Students Patterns of Discourse

Patterns of facilitator/student interactions based on within note discourse analyses of Groups 1

and 2 across 40% of the dataset— specifically all notes in Mr. Singh’s Pain, Part 2 and Judy’s

Last Days, Part 2—were scored by the researcher. Two of the four modules were scored by a

second rater. Lack of clear definition of categories led moderate interrater agreement (Mr. Singh,

Group 1 Kappa = 0.446 with p < 0.000; Group 2 Kappa = 0.566 with p < 0.000). Tabak and

Baumgartner’s (2004) categories of monitor, mentor and partner were expanded to include the

facilitator stance of expert, characterized by didactic teaching; in-between categories of

mentor/partner and partner/expert were also added. The results are presented below.

7.3.1.1 Patterns of facilitator discourse. One of the novel findings in this study is the high

level of facilitator participation in the discourse. Although we have determined that facilitators

were central to the social network, we do not know how they participated and what

sociocognitive facilitator/student dynamics resulted. Results of analysis of patterns of

facilitator/participant discourse patterns, characterized as facilitator stance, are indicated in Table

47. A series of characteristics defined the stance of discourse and was used in analysis and

scoring.

Table 47

Facilitator/Participant Discourse Patterns: Discourse Stance

Facilitator/Participant Discourse Patterns

Monitor Mentor Mentor/Partner Partner Partner/Expert Clinical

expert

Group 1, Facilitator 1

Mr. Singh’s Pain, Pt.2

(n of notes = 31/37)

3 7 1 8 8 4

Judy’s Last Days, Pt. 2

(n of notes = 20)

2 3 0 5 7 3

Group 2, Facilitator 2

Mr. Singh’s Pain, Pt. 2

(n of notes = 26)

7 4 0 4 6 5

Judy’s Last Days, Pt. 2

(n of notes = 13)

1 2.5 1 2 1 5.5

Totals 13 16.5 2 19 22 17

188

Results of pattern counts in Group 1 across the two modules examined are strongest in the

categories of facilitator as mentor, partner, and partner/expert. In Group 2 pattern demonstrates

highest scores in categories of facilitator as monitor, and as partner/expert and clinical expert.

Collectively all facilitator roles are acknowledged. This demonstrates the necessary flexibility of

skills required by facilitators and the range of roles necessary to support online work. These

patterns are both indicative of facilitator/student symmetry and asymmetry discourse patterns

and shifts between contributory and authoritative stance. This is a significant departure from

traditional stance of instructor as clinical expert in CME. Although this role is acknowledged in

online collaborative knowledge work, it is not the dominant role in three of the four modules.

Facilitator 2 uses this role predominantly in the last module to do much didactic teaching.

Facilitation of the two Group 1 modules are strongly characterized by more democratic discourse

enabled by greater use of more symmetric roles and less use of the asymmetric, didactic role of

the clinical expert.

Highest counts for Facilitator 1 were in categories of partner and partner/expert that is

characterized as inquiry in design and belief modes. Highest count for Facilitator 2 was the role

of monitor, which is characterized by administrative inquiry. Lower count for monitoring by

Facilitator 1 is likely due to differences is student characteristics; that is, Group 1 participants

demonstrated more agency and therefore needed less monitoring to participate than students in

Group 2.

Tabak and Baumgartner (2004) found the role of teacher-as-partner important to identity

formation and inquiry-based science learning in classrooms with young students. In this study

facilitator as partner and expert emerged as strong roles supporting online inquiry and knowledge

work. Further examination of discourse patterns is required to understand the moves and the

process of knowledge improvement. Thus, patterns of discourse statements and questions are

analyzed in the next section.

7.3.1.2 Patterns of discourse statements and questions. Didactic teaching is often

characterized as knowledge telling—a one-way process conveying information through a series

of statements. This next analysis was driven by the hypothesis that questions, on their own or

along with statements, drive Knowledge Building. It can be said that the relationship between

questions and statements scaffold knowledge growth and drive abductive and adductive

knowledge work, more broadly and deeply. Therefore, in this next analysis, evidence of

189

questions (through numeric count) was established, along with the relationship between

questions and statements. Notes were categorized by the researcher as:

· Statement followed by a question;

· Question only;

· Question followed by a statement;

· Statement only; or

· Administrative/other

In addition, students-at-the-core and the facilitator at-the-core (based on previous social network

position/power analysis) were examined to compare question/statement patterns (Table 48).

Table 48

Patterns of Discourse Statements and Questions by Facilitators and Students,

and Student/Facilitator Patterns at the Core

Note. S = Statement. Q = Question. Total = Question Only + Statement followed by Question + Question followed

by Statement. Facilitator-at-the-core + All Students = ALL.

S followed

by Q

Q only

Q followed

by S

Total Q

+ Q/S

S only

Admin.


Group 1

Student-at-the-core (n = 1) 7 0 0 7 11 0

Facilitator-at-the-core (n = 1) 5 6 2 13 14 6

All Students 18 5 0 25 37 3

ALL 23 11 2 36 51 9

Group 2




ALL 10 3 2 15 35 8


Group 1

Student-at-the-core (n = 3)* 4 0 0 4 17 0



ALL 13 1 3 17 34 6

Group 2




ALL 8 0 1 9 15 4

190

Results of this count and discourse pattern analysis indicate that students at the core of the social

network most frequently made statements or made statements followed by a question(s). In the

modules examined here in there were no instances of students at the core posting a note in KF

with only a discourse question or a question followed by a statement. Similarly, facilitators at the

core most frequently made statements and made statements followed by a question within the

same note. However, facilitators also demonstrated other discourse patterns of posting questions

only and questions followed by statement(s) in a KF note.

Group 1 students and facilitator at the core posted more questions that those in Group 2. In Mr.

Singh’s Pain, Part 2, the facilitator for Group 1 posted six questions while the facilitator for

Group 2 posted one question. In the same modules the Group 1 facilitator cumulatively posted 13

questions in all categories involving questions while the Group 2 facilitator only posted six.

Similar discourse patterns were found for all students. In all modules examined students posted

statements most frequently. In three of the four modules students posted notes with statements

followed by a question(s) approximately 30% to 50% as often as notes with statements only.

These discourse patterns are striking in that collective knowledge work proceeds not by

knowledge telling through statements alone but by inclusions of questions with statements or by

questions alone. Discourse patterns including questions are highly prevalent, particularly

statements followed by questions by both facilitators and students. Thus, we can clearly see that

student and facilitator questions play an important role in advancing knowledge work and can be

considered a demonstration of intentional Knowledge Building and epistemic agency to improve

ideas. However we do not know how these questions and statements are related to the interaction

of ideas—to work in belief-mode and design-mode. Some questions are intended only to clarify

facts, like how to titrate dosages of pain medication, and are based on the predetermined learning

objectives posted in the module. Other kinds of questions are defined as those linked to new and

emergent ideas intended to drive Knowledge Building deeper, adductive Knowledge Building or

expand ideas more broadly, abductive Knowledge Building.

The next analysis was aimed at determining whether participant discourse and ideas remained

within the constraints of the predetermined learning objectives in each case-module or if

emergent ideas above and beyond the predetermined learning objectives were present in the

discourse, and if so, what are the emergent themes identified.

191

7.3.2 Content Analysis of Themes, Beyond the Predetermined Learning Objectives

7.3.2.1 Content analysis of knowledge work. This next set of analyses by the researcher

employs the same four modules used previously for within-content analysis. The focus of these

analyses was to determine if within note discourse patterns demonstrated belief-mode and

design-mode knowledge work through emergent ideas. The objective was to obtain an

understanding of the relationship between predetermined knowledge objectives (set forth by the

course content designers), in terms of how participants in the problem space worked with these

objectives. The question was: Is there evidence of knowledge work above and beyond these

objectives, that could be distinguished from the learning objectives and identified as an emergent

idea and/or theme, consistent with the Knowledge Building notion of design-mode and belief-

mode work? Within-notes analysis of the problem space was conducted. Social network

knowledge work by students and facilitators were examined.

Results indicated that less than 49% and 48% of knowledge work in Group 1 across both

modules related to predetermined learning objectives and 51% and 52% of the discourse was

around emergent ideas, beyond those listed in the learning objectives (Table 49).

In contrast, the majority of Group 2 notes were knowledge work related to the preidentified

learning objectives. In Mr. Singh’s Pain, Part 2, 57.63% of knowledge work with notes pertained

to the learning objectives. Similarly in the last module, Judy’s Last Days, Part 2, 73.33% of

participants worked with the predefined learning objectives. Only 41.67% and 33.33% of

participants in the problem space identified emergent issues for knowledge work (Table 49).

Table 49

Results of Knowledge Work With Predefined Learning Objectives or Emergent Ideas

Total n of notes n of notes related to

objectives (%)

n of notes beyond

objectives/emergent ideas (%)

Group 1

Mr. Singh’s Pain, Part 2 106 52 (49.07) 54 (50.94)

Judy’s Last Days, Part 2 64 31 (48.44) 33 (51.56)

Group 2

Mr. Singh’s Pain, Part 2 59 34 (57.63) 25 (41.67)

Judy’s Last Days, Part 2 30 22 (73.33) 8 (26.67)

Note. n = number.

192

Emergent ideas beyond learning objectives were identified across both groups and in all

modules. Work with ideas defined by the learning objectives and work with emergent ideas

beyond the learning objectives was very balanced by Group 1. Thus, we can conclude that in this

study, knowledge work proceeds in belief and design-mode and that there is a relationship

between these modes of work. Knowledge work of this nature seems to involve abductive,

broadening of ideas, or adductive, deepening ideas to improve understanding and may help to

explain why Group 1 experiences greater knowledge gains. In the next analysis themes were

identified through the categorization of discourse threads.

7.3.2.2 Relationship of themes, threads, and learning objectives in the social network

discourse. Knowledge Building through belief-mode and design-mode or adduction and

abduction of knowledge and ideas can be connected to content themes through categorization of

threads (Zhang, et.al, 2009). Themes, in turn can be identified as emergent or related to

predetermined learning objectives. The results of this analysis are presented below and

summarized in Table 50.

Themes identified in discourse threads related to knowledge work based on the predefined

objectives in Mr. Singh’s Pain, Part 2, which were:

· Pain management

· Opioid use

· Neuropathic pain

· Bone pain

Knowledge work in belief and design-mode work in Mr. Singh’s Pain, Part 2, was characterized

by adductive and abductive knowledge work with new ideas. Nine new themes emerged in the

Group 1 discourse and were identified as:

· Personal cases

· Authentic practice

· Cost of drugs

· In-practice or “how-to” details

· Local access to palliative nursing care and support services

· Dealing with personal emotions

· Communication with patients and family members

193

· Better understanding of various cultural and religious practices

· Self-care—how to care for oneself as a palliative care physician

Participants identified many important ideas for discourse and as indicated above. Authentic new

themes emerged beyond the predetermined learning objectives. Analogies from the case of Mr.

Singh were made to personal cases from physicians’ own practices and authentic issues from

their own experiences. In Group 1, 4 students provided personal cases to broaden the problem

space.

In Group 2, one student contributed a personal case from their own practice to the problem

space. This type of emergent contribution shifts the discourse and enables students to become co-

designers of the curriculum, metadesigners of the problem space. Thematic topics, content, and

ideas of importance to them are thus virtually added to the predetermined objectives defined by

the curriculum designers. Some themes identified by Group 2 were the same as those in Group 1;

others were not—new themes have been italicized below. Group 2 themes include:

· Personal cases

· Authentic practice

· Local access to palliative care support (e.g., community nurses)

· Emotions

· Self-care

· Cultural practices

· Communication with patients and families

· Education of families

· Coverage

· Funerals

Analogies to personal experiences are common throughout this discourse. The emergence of

discourse around how treating physicians can deal with emotions and whether or not to attend a

patient’s funeral are important professional practice ideas that are well beyond the learning

objectives originally identified by curriculum designers for this module—but perhaps the most

important. These are themes and ideas rarely found in typical continuing medical education

course and clinical references on palliative care.

194

Themes identified in discourse threads related to predefined objectives and knowledge work in

Judy’s Last Days, Part 2 were:

· Management of last hours of life

· Family concerns and counseling

· Signs of imminent death

· Symptoms

Similar to the previous modules examined, discourse in Judy’s Last Days, Part 2, clearly

demonstrated numerous emergent issues and ideas, categorized as themes and threads of beyond

the learning objectives.

Table 50

Summary of Emergent Themes/Threads Beyond Learning Objectives


Learning objectives Emergent themes/threads in abductive and adductive KB beyond objectives

1. Pain management

2. Opioid use

3. Neuropathic pain

4. Bone pain

Group 1

(N of notes =

106)

Authentic practice

Group 2

(N of notes =

59)

Authentic practice

Personal cases Personal cases

Drug cost Support/Emotions

In-practice Funerals

Local access Local Access

Emotions Coverage

Communication Communication

Culture/Religion Culture

Self-care Educate Family

Self-Care

1. Management of last

hours of life

2. Family concerns and

counseling;

3. Signs of imminent

death

4. Symptoms

Group 1

(Number of

notes = 64)

Authentic practice

Group 2

(Number of

notes = 30)

Authentic practice

Personal cases Personal cases

Culture/Religion Funerals

Teamwork Culture/Religion

Homecare Home visits

Local Access Death certificate

Suffering Billing/OHIP codes

Communication

Death certificate

Self-care

Emotions

In conclusion, codesign of the curriculum becomes deeper and broader through student

contributions of analogous cases, authentic experiences, and identification of personal

knowledge gaps. Group 1 identified 11 themes beyond the predetermined objectives; some were

similar to those from Mr. Singh’s Pain, Part 2, but intentionally further broadened ideas of

195

discourse. Again personal cases from individual physicians’ private practices were introduced to

into the problem space. Philosophic and clinical discussions around the concept of “suffering”

emerged as a central issue. Group 2 identified 7 emergent themes; some overlapped those

identified by Group 1 and also included student contributions of their personal practice cases.

Both groups identified professional practice gaps in knowledge around how to issue a death

certificate, billings and OHIP codes.

7.3.2.3 Summary of emergent themes and metadesign results. It is clear from the foregoing

analysis that numerous important themes, beyond the predefined learning objectives for these

modules, emerge in the discourse. Emergent themes are different according to Groups, and the

students in those groups and the facilitator that facilitates them.

Student level of knowledge, lack of knowledge, interests, and other things that usually have no

place in a curriculum, like physician emotions around death and dying, become artifacts of

discourse within the problem space for design-mode and belief-mode work in Knowledge

Forum. Facilitator centrality in the social network, is flexible, demonstrated by shifting roles of

mentor, partner, and expert, enabling corresponding student shifts toward democracy in the

discourse. Participant clinical discourse was found to be at a very high level, characterized as

elaborated and compiled.

It was posited that questions drive Knowledge Building and results indicated that questions are

contributed to the discourse by students, particularly those at the core of the social network;

questions are not the sole purvey of the facilitator. The foregoing analyses demonstrated strong

social network content patterns and a positive relationship between design-mode knowledge

work and new ideas that I have called abductive and adductive knowledge work. Abduction

intentionally broadens knowledge by explaining more phenomena, for example in this study

students contributed analogous cases to the discourse space to broaden understanding of patients

symptoms during the last days of life. Working more and more deeply with facts can be thought

of as both adductive and abductive Knowledge Building. Adduction can be defined as the

intentional deepening of understanding by investigation of deeper layers and/or greater details.

An example of abductive and adductive knowledge work from the discourse is clarification of

titration of opioids and discussion around the nuances of opioid treatment as the “art and

science” of pain management.

196

Strong evidence in this study supports and elucidates the complex aspects of Knowledge

Building. Online design of this continuing medical education program in Knowledge Forum was

both structured/prescribed by expert-driven objectives and intentionally unstructured for

emergent knowledge work to be determined by participants, students and facilitator, to enable

both belief-mode and design-mode Knowledge Building.

Thus, it can be said that students in this continuing medical education course have had the

opportunity to participate in the collaborative design of their knowledge work and to imbue the

problem space with their concerns, authentic problems, and real-world practice issues, as well as

highly personal and ethical facets, not typically brought into the discourse of traditional medical

teaching environments. In essence they have worked to advance community knowledge and

through their emergent ideas they have also contributed to the metadesign of the curriculum. I

have termed this type of Knowledge Building metadesign democratic innovation for continuing

medical education. Democratic innovation includes not only sociocognitive participation, but

emergent codesign of curriculum—work typically assumed by teachers and experts, not by

participants.

7.3.3 Social Network Analysis of Complexity of Discourse

The results below delve into the complexity of discourse and are categorized according to the

Bordage scale of complexity of clinical discourse (1991). The same four modules were used for

analysis from Group 1 and 2, Mr. Singh’s Pain, Part 2 and Judy’s Last Days, Part 2. This

represents 40 % of the dataset which was scored by the principal investigator; three of the four

modules were scored by a second rater. Interrater reliability was calculated using Cohen’s Kappa

coefficient. Interrater agreement was moderate (Mr Singh, Group 1, Kappa = 0.474 with p <

0.000; Mr Singh, Group 2, Kappa = 0.504 with p < 0.000; Judy’s Last Days, Group 2, Kappa =

0.427 with p < 0.000). Categories of elaboration and compilation may have been too similarly

defined.

Bordage’s scale of semantic analysis of clinical discourse (Bordage et al., 1991) was used to rate

complexity of discourse within notes. The 4-point scale categorizes complexity of clinical

discourse as reduced, dispersed, elaborated or compiled. These terms are defined as follows:

· Reduced discourses, typically characterized by little exposition of thought, often

limited or inaccurate knowledge.

197

· Dispersed discourses, typically characterized by numerous often unrelated diagnostic

possibilities, both reasonable and unreasonable; thought processes are not organized.

· Elaborated discourses, characterized by the use of SQs (semantic qualifiers) to

compare and contrast diagnoses, either explicitly or at times, implicitly; the

clinician’s train of thought is clear.

· Compiled discourses, characterized by structured, concise, and condensed syntheses

of case data using selected SQs (semantic qualifiers) to clearly “rule in” and “rule

out” pertinent diagnostic possibilities; most often found in the discourses of

experienced, expert clinicians. (Bordage et al., 1991, p. 3).

Case and online library notes created by the curriculum design team were not scored or included

in this analysis. There were no clinical discourse notes rated as reduced or dispersed. However,

there were some administrative notes that were categorized and placed these categories.

Examples of reduced/dispersed administrative discourse are notes containing notification of

online tests, reminders of dates for next modules, communication about participant holidays, etc.

In the last module, Judy’s Last Days, Part 2, “other” types of notes emerged toward the end of

the course; these were thank you notes, reflections on the course and on practice. These “other”

notes were categorized as reduced/dispersed and put together with the administrative notes.

Social network analysis of complexity of discourse was similar across all modules in that almost

all student and facilitator discourse was rated as complex or elaborated. Since all participants in

this CME study are family physicians, it is not surprising that all clinical discourse reflects a high

level of complexity, within notes, across all four modules scored (Table 51).

It is noteworthy that facilitator elaborated discourse notes are more prominent than complied in

three of the four modules. Group 1 Facilitator wrote approximately 30% more elaborated notes

than compiled. Group 2 Facilitator wrote approximately 30% less elaborated notes than compiled

notes in Mr. Singh’s Pain, Part 1; and more than 50% more elaborated notes than compiled notes

in Judy’s Last Days, Part 2.

In contrast, student note complexity, in three of the four modules, is almost evenly distributed

between compiled and elaborated discourse. Only students in Group 1, in Judy’s Last Days, Part

2, exhibited 15.63% compiled discourse and 40.63 elaborated discourse complexity. However, it

can be said that both students and facilitators demonstrated a high level of Knowledge Building

198

work that was well -distributed and demonstrated agency to compile and synthesize ideas, as

well as to explore and elaborate their understandings.

Table 51

Semantic Analysis Results of Complexity of Discourse

2008/2009 Complexity of discourse results

Group 1 Mr. Singh’s Pain, Part 2

(n of notes = 106)


(n of notes = 64)

Compiled

% (n)

Elaborated

% (n)

Admin. (n) Compiled

% (n)

Elaborated

% (n)

Admin./

Other (n)

Students

(n = 10)

30.19

(32)

32.08

(34)

2.83

(3)

15.63

(10)

40.63

(26)

12.5

(8)

Facilitator

(n = 1)

11.32

(12)

17.92

(19)

5.66

(6)

10.94

(7)

14.06

(9)

6.24

(4)

ALL 41.51

(44)

50

(53)

8.49

(9)

26.56

(17)

54.69

(35)

18.75

(12)

Group 2 Mr. Singh’s Pain, Part 2

(n of notes = 59)


(n of notes = 30)

Compiled

% (n)

Elaborated

% (n)

Admin. (n) Compiled

% (n)

Elaborated

% (n)

Admin./

Other (n)

Students

(n = 9)

27.12

(16)

27.12

(16)

1.69

(1)

20

(6)

20

(6)

13.33

(4)

Facilitator

(n = 1)

16.64

(11)

11.86

(7)

13.56

(8)

10

(3)

26.66

(8)

6.66

(2)

ALL 45.76

(27)

38.98

(23)

15.25

(9)

30

(9)

46.66

(14)

23.33

(7)

Although we now know that the discourse was complex and emergent and students worked

intentionally to improve knowledge in belief and design mode, we have not verified that

principles of Knowledge Building were evident in the discourse. Results of this analysis will be

provided next.

7.3.4 Evidence of Knowledge Building Indicators Within Social Network Discourse

The final facet of results to be presented is the representation of all Knowledge Building

principles within the discourse of these four modules. The same four modules were scored on a

5-point scale (Sibbald, 2009) indicating degree to which each module demonstrated each of the

12 Knowledge Building principles as defined by Scardamalia (2002). The principle of pervasive

Knowledge Building was scored only in relationship to the context of this program since it is not

199

known if Knowledge Building occurred beyond this program in other aspects of participants’

work and lives. The four modules were scored by the researcher and a second rater. Interrater

reliability was high (83%). The two raters agreed 10 out of 12 times.

All Knowledge-Building principles were evidenced in each module scored; many were found to

be strong and consistent, as indicated in Table 52. Principles of community responsibility,

democratic knowledge, idea improvement, diversity, Knowledge-Building discourse, and work

with real ideas/authentic problems were prominent. Indicators of epistemic agency and rise-

aboves rated more cautiously to acknowledge broader context of ideas beyond the local use.

Themes of emergent ideas were clearly over and above learning objectives, and clearly

demonstrated metadesign knowledge work.

Table 52

Knowledge Building Principles Demonstrated in the Discourse

Knowledge Building Principle

Strong and

consistent

Relatively

strong and

consistent

Observed but

inconsistent

Occasional

Not

observed

Real ideas, authentic problems x

Knowledge Building discussion x

Improvement of ideas x

Diversity of ideas x

Conceptual change:

understanding that rises above

previous ideas

x

Epistemic agency x

Community knowledge/

Collective responsibility x

Democratizing knowledge x

Symmetric knowledge

advancement x

Constructive use of authoritative

sources x

Embedded transformative

assessment (through discourse) x

Pervasive Knowledge Building x

200

Group 1 in particular provided numerous instance of emergent ideas. As previously

demonstrated, emergent ideas formed nearly 50% of the discourse and provided clear

demonstration of design-mode work and metadesign of the curriculum. A series of notes from

the 2008/2009 discourse are provided below as evidence and exemplars of Knowledge Building

principles of community knowledge, collective responsibility, working to improve ideas,

democratic and emergent knowledge work.

The first example (Figures 71, 72, 73) is a series of three discourse notes from Mr. Singh’s Pain,

Part 2. In the first note the student identifies a knowledge gap, her lack of understanding around

a complex issue that may not have a “right/wrong” answer. Epistemic agency is demonstrated

through her identification of the issue and her attempt to find the answer to this question by

reading various references; still fuzzy, the student posts questions to the problem space. The

student exemplifies “work at the edge of knowledge” (Scardamalia, 2003) demonstrating the

Knowledge Building principle of “constructive use of authoritative sources” both through

accessing written references and the KF problem space, as well as internal assessment,

identifying an important problem of interest to the community, exemplifying the principle of

“embedded and transformative assessment.” The facilitator responds didactically and then adds

insight based on her own professional practice. The third note, by another student, recognizes the

initial student’s epistemic efforts. In many ways, this interchange is characteristic of “rise-above”

Knowledge Building, work with higher-level formation of problems that involves complex and

messy ideas, but result in moving to a higher plan of understanding (Scardamalia, 2002).

Figure 71. Exemplar 1: 2008/09, Group 1, Mr. Singh’s Pain, Part 1.

201



202

In the next example (Figures 74–85), the interchange begins with a series of questions from the

facilitator. The build-on responses are numerous and include two personal practice case

analogies that extend the discourse. The flow of the discourse evolves into spirituality issues,

advanced directives, practical issues of how to pronounce and fill out a death certificate,

logistical issues, questions about attending patient funerals, and reflections on “Why we do it.”

This powerful discourse is triggered by the facilitator and then taken over by the students—one

student emerges as the leader and takes over the role of facilitator; this is one of the students

positioned at the core. This interaction provides strong exemplars of numerous Knowledge

Building principles, including democratizing knowledge, real ideas/authentic problems,

improvable ideas, diversity, epistemic agency, rise-above, community knowledge/collective

responsibility, symmetric knowledge advancement, involving both belief-mode to advance the

edge/deepen understanding and design-mode work to broaden knowledge and experience.

Figure 74. Exemplar 4: Group 1, Judy’s Last Days, Part 2.

203


204



205



206



207


New Note:


208



209

At the conclusion of the last module, Group 1 students choose to voice their opinions about the

course and reflections on the process (Figures 86–89). The strength of the Knowledge Building

community and appreciation for opportunistic collaboration and metadesign rings clear in the

notes below.

Figure 86. Exemplar 16: Group 1, Judy’s Last Days, Part 2—Reflections


210



7.3.5 Cluster 3 Summary of Social Network Position and Power Analysis, Sociocognitive

Dynamics, and Indicators of Knowledge Building

This cluster was guided by the investigations of the following research subquestion: What are the

social network relationships between structural position, power (defined as centrality of ideas),

and knowledge improvement; and how are these relationships reflected in facilitator/student

sociocognitive dynamics of Knowledge Building (beyond learning objectives), through emergent

themes, in complexity of discourse, and by indicators of Knowledge Building?

211

7.3.5.1 Sociocognitive dynamics of social network position/power.

7.3.5.1.1 Facilitator/students patterns of discourse patterns. Strongest Group 1 Facilitator

discourse pattern is as partner and partner/expert; strongest Group 2 Facilitator discourse pattern

is as monitor and partner/expert. In regards to statement/questions patterns, students and

facilitators at the core of the social network most frequently made statements or made statements

followed by a question(s). Overall, student/facilitator discourse patterns are striking in that

collective knowledge work proceeds not by interchange of statements alone; but most often

through statements followed by a question(s). Group 1 Facilitator posed more than 50% more

questions in both modules examined, than Group 2 Facilitator.

7.3.5.1.2 Results of analysis of complexity of discourse. Social network analysis of

complexity of clinical discourse was similar across all modules in that all student and facilitator

discourse was rated as complex or elaborated. No discourse was rated as dispersed or reduced.

Since all participants in this study are family physicians, it is not surprising that all clinical

discourse reflects a high level of complexity, within notes, across all four modules scored.

7.3.5.1.3 Emergent themes and metadesign knowledge work (beyond predefined learning

objectives). Emergent themes were identified in Group 1 and Group 2 discourse. Emergent

knowledge work can be characterized as adductive and abductive Knowledge Building, working

to improve knowledge so that it is understood more deeply and broadly. Group 1 and Group 2

identified different themes for knowledge work according to their needs and interests. Analogous

cases, “my experience,” “in my practice,” emotions, and pragmatics of practice (e.g., OHIP

billings). Personal knowledge lacks were identified, “need to know” questions were posted,

misconceptions corrected, and grey areas discussed that have no one answer (e.g., the art and the

science of palliative care practice).

7.3.5.1.4 Results of Knowledge Building indicators within social network discourse. All

Knowledge Building principles were evidenced in each module scored and most were found to

be “strong and consistent.” Exemplars of principles of community responsibility, democracy,

improvable ideas, epistemic agency demonstrated primarily through emergent themes/ideas, are

indicative of metadesign Knowledge Building.

212

Because of the complexity of results of this study a comprehensive overview of all results will be

provided next to ensure that the reader has an understanding of all details and the flow of results

to gain the big picture of the relationships across findings. Results will be further elaborated

upon in the next chapter, the Discussion chapter. Study conclusions, will also be framed by the

three clusters of research subquestions: (a) traditional outcome measures; (b) performance “over

and above” traditional measures (beyond learning as traditionally conceived and measured); and

(c) questions regarding the sociocognitive dynamics that enable work over-and-above traditional

learning.

7.4 Overview of Results

Design of the online environment and evaluation tools are validated in the 2004/2005 pilot study.

Minor changes were made, particularly to pain pre-/posttest structure for future iterations.

Traditional assessment measures, including pre-/posttests, attitude and opinion surveys, and

online activity statistics are used to assess outcomes from 2005 to 2008. Assessment is expanded

to include social network and Knowledge Building analyses of the 2008/2009 discourse space.

Results of all findings are summarized below.

7.4.1 Pilot Study 2004/2005

Key results from the pilot study were:

· Pre-/posttest was validated. Specific questions were refined to improve reliability.

· Online activity over five modules of percent of notes read was high ranging between

74-85%; number of total notes contributed was 306 and percent of notes lined were

strong, between 19% and 72%.

· Attitudes and opinions of were very positive. Overall, 80% of respondents rated the

Knowledge Building component of this program as above average or excellent.

7.4.2 Cluster 1: Traditional Measures

7.4.2.1 Matched results of pain knowledge pre-/posttests, 2005–2009.

· 2005/2006 total mean score across all objectives (1 to 4) demonstrated a significant

improvement in pain knowledge of 8%, from 73% on pretest to 81% on posttest, on

paired t-test = 3.48, p < .006. Cohen’s d demonstrated a large effect size of .87.

213

· 2006/2007 total mean score across all objectives (1 to 4) demonstrated strong

improvement in pain knowledge of 10%, from 68% on pretest to 78% on posttest;

however this was not significant (probably due to small sample size) on paired t-test =

2.15, p < .075; Cohen’s d demonstrated a large effect size of .93.


improvement in pain knowledge of 7%, from 68% on pretest to 75% on posttest, on

paired t-test = 2.22, p < .04. Cohen’s d demonstrated a medium effect size of .64.


improvement in pain knowledge of 14%, from 67% on pretest to 81% on posttest, and

on paired t-test = 4.30, p < .001. Cohen’s d indicated a very large effect size of 1.15.

· 2005–2009 cumulative total mean score across all objectives (1 to 4) demonstrated a

significant improvement in pain knowledge of 9%, from 69% on pretest to 78% on

posttest, and on paired t-test = 5.34, p < .001. Cohen’s d indicated an overall large

effect size of .82.

7.4.2.2 Attitude and opinion results 2005–2009

Online collaborative Knowledge Building was rated above average/excellent by 86.7%

respondents. Ninety-two percent agreed/strongly agreed that they would like to see more

distance education program for CME.

7.4.3 Cluster 2: Performance Over and Above Traditional Measures

7.4.3.1 ATK measures 2005–2009.

· In 2005/2006 participants read a mean of 86.3% of notes across all modules and


· In 2006/2007 participants read a mean of 92.3% of notes across all modules and


· In 2007/2008 participants in Group 1 read a mean of 92.6% of notes and wrote an

average of 30.93 notes per module; participants in Group 2 read a mean of 92.4% of

notes across all modules and wrote an average of 18.25 notes per module; and

· In 2008/2009 participants in Group 1 read a mean of 79.33% of notes and wrote an

average of 36.82 notes per module; while participants in Group 2 read a mean of

214

85.7% of notes across all modules and wrote an average of 25.1 notes per module.*

This year was selected for further social network analysis.

· Facilitator note contributions and build-ons more than twice that of most students;

· SNA of who built-on whose notes is fan shaped with facilitator as pivotal point and

arch shaped indicative of students building-on each other’s ideas; network edges =

25;

· Number of notes read and SNA of who read whose notes was well distributed;

network edges = 110

7.4.3.2 Social network results (KF analytic tools) 2008/2009: Groups 1 and 2.

7.4.3.2.1. Mr. Singh’s Pain, Part 1: Group 1.

· Facilitator note contributions and build-ons approximately twice that of most

students, except one;

· SNA of who built-on whose notes is becoming more distributed around the fan shape

with facilitator as pivotal point; network edges = 27;


network edges = 100

7.4.3.2.2 Mr. Singh’s Pain, Part 2: Group 1.


students, except one;




network edges = 100

7.4.3.2.3 Mary’s Misery: Group 1.

· Facilitator note contributions and build-ons approximately twice that of students;




network edges = 91.

215

7.4.3.2.4 Judy’s Last Days, Part 1: Group 1.

· Facilitator note contributions and build-ons more than three times that of most

students; reading measures are high for all participants;

· SNA of who built-on whose notes fan shaped with facilitator as pivotal point;

network edges = 14;


network edges = 76



reading measures are high for all participants;

· SNA of who built-on whose notes fan shaped with facilitator as pivotal point;

network edges = 20;


network edges = 66.



only 8 people are building-on but 12 are reading.

· SNA of who built-on whose notes is somewhat distribute; network edges = 18;


network edges = 67


· Facilitator note contributions and build-ons more than three times that of most

students.

· SNA of who built-on whose notes is distributed in an arch pattern; network edges =

21;


network edges = 55

7.4.3.2.8 Mary’s Misery: Group 2.

· Facilitator note contributions and build-ons was approximately twice that of most

students; reading of notes was high for almost all participants.

216

· SNA of who built-on whose notes is somewhat distribute; network edges = 12;


network edges = 54


· Facilitator note contributions and build-ons more than twice that of most students.

· SNA of who built-on whose notes is distribute amongst few participants; network

edges = 6;


network edges = 17



students; reading level was high.

· SNA of who built-on whose notes is distribute amongst few participants; network

edges = 7;

· Number of notes read and SNA of who read whose notes was well distributed among

the few remaining participants; network edges = 31

7.4.3.2.11 Results of 2-way ANOVA on 2008/2009 Groups 1 and 2. Overall, the difference

between the 2008/2009 matched pre- and posttest means combined over group was significant, F

(1,16) = 17.97, p < .001. Overall difference between groups was not significant. Both groups

were strong; both groups demonstrated a similar effect size, of 1.11 and 1.12 respectively.

However, Group 1 demonstrated greater increase and higher level knowledge gains than Group

2. Group 1 demonstrated a 16% improvement in total pain knowledge, from 68% on pretest to

84% on posttest. In comparison Group 2 scores increased 12% from 66% on pretest to 78% on

posttest. Although both groups were strong, Group 1 was deemed to be stronger.

7.4.3.2.12 Network links and density measures and tests of significant difference and

effect size. Comparison between groups (with facilitator) was conducted. Groups 1 mean

network density of build-ons (0.278) was higher than Group 2 (0.182). Group 1 network density

measures of notes read (0.798) was higher than Group 2 (0.657). When facilitator was removed,

SNA results indicated groups remained strong in network build-ons and reading. On t-test and

analysis of effect size, results demonstrated that there was no significant difference on density or

217

effect size of notes build-on and read with and without facilitator. Therefore, student

participation was strong and self-sustaining.

7.4.3.2.13 Social network measures of centrality: Eigenvector, in-degree, and out-degree;

and results of t-test and effect size. Mean Eigenvector centrality measures of build-ons and notes

read are low in both Groups 1 and 2. These low measures are indicative of well-distributed

groups. Eigenvector analysis of groups with and without facilitators demonstrates little change

and is therefore indicative of strong student networks. In-degree/out-degree measures of Group 1

and Group 2 build-ons dropped markedly in some modules when the facilitator was removed

indicating good distribution across the student population, yet high degree of facilitator

participation.

Significant difference and large effect size were found for Group 1 and 2 when tested with and

without Facilitator, on social network centrality dimensions of in-degree and out-degree of notes

built-on. Therefore we can conclude the Facilitators in Groups 1 and 2 were highly participative

in terms of receiving and linking or building-on others notes and had a large effect.

7.4.3.2.14 Significant difference between groups in social network density measures.

Comparison results between Groups 1 and 2, with a facilitator, across all five modules indicates

that density of notes built-on is significantly different. Difference in density for notes built-on

was significantly stronger in Group 1 (p<.01) and effect size was particularly large measuring,

1.31. Density of build-on notes appears to be the major factor emerging from SNA in this case

study that is related to higher levels of knowledge.

Further SNA analyses only focused on build-ons notes (not notes read) including facilitator since

differences were found across groups; notes read measure was very high for both groups and

showed little difference.

7.4.3.2.15 Clique members and clique cohesion index. Group 1 had almost twice as many

total cliques (27) as Group 2 (14). Most striking was the fact that the Group 1 facilitator

participated in all 27 cliques, which is more than three times as many as the Group 2 facilitator,

who participated in 8 of 14 cliques. Despite the very high level of the Group 1 facilitator’s

engagement, the cohesion index across modules was lower than Group 2, indicative of a more

highly distributed and inclusive social networks. Interclique connectivity was maintained in both

groups by the numerous participants that were active across many cliques. “This bridging

218

phenomenon provides for a wealth of information flowing to all members” (Aviv et al., 2003, p.

13).

7.4.3.2.16 Relationship of SN structural analysis of three pain modules to pre-/posttest

knowledge improvement scores. When Groups 1 and 2 were compared, relationship patterns

indicated that higher performance on Group 1

pre-/post knowledge tests were related to higher levels of build-on notes in KF, higher number of

network edges, significant difference (p < .001) in density of build-on notes, lower cohesion

index and centrality measures, as well as a greater number of cliques and facilitator engagement,

in an opportunistic and evenly distributed network.

When Groups 1 and 2 were combined for Spearman nonparametric correlations of significance

of the posttest score with the social network centrality variables, it was found that of the nine

correlations, all are positive and all are above .2, six above .3, and four are significant. In-degree,

out-degree, and Eigenvector centrality measures in various pain modules were significantly

correlated with posttest scores.

7.4.4 Cluster 3: Social Network Analyses and Sociocognitive Dynamics That Support

Knowledge Building Over and Above Learning

7.4.4.1 Results of social network position/power analysis 2008/2009. Network position

(centrality of facilitator/students and distribution of students) and relationship to network power

(prestige and dominance/democracy) although related to SN structural analysis is considered

separately and so that results can be more clearly linked to social network semantic analysis.

The greatest “gainers” on pain knowledge pre-/posttest in the Group 1, shifted positions over

modules and were positioned in the midfield and the periphery more often that in the core; the

greatest “gainer” in Group 2 never occupied a position at the core—only in the mid and

peripheral fields. It appears that there is much to be gained from work at mid field and periphery

positions in a CME context.

It was posited that students with the highest scores on pain pretest would often assume the core

position, sharing this position with the facilitator, which in fact was the case for some but not all.

7.4.4.2 Results of social network content analysis 2008/2009. Results summaries of the

following series of within note analyses are presented below:

219

· Facilitator/students patterns of discourse;

· Emergent themes of discourse (beyond learning objectives);

· Social network complexity of clinical discourse

· Results of analysis of Knowledge Building indicators and exemplars.

7.4.4.2.1 Facilitator/students patterns of discourse patterns. Strongest Group 1 Facilitator

discourse pattern is as partner and partner/expert; strongest Group 2 Facilitator discourse pattern

is as monitor and partner/expert. In regards to statement/questions patterns, students and

facilitators at the core of the social network most frequently made statements or made statements

followed by a question(s). Overall, student/facilitator discourse patterns are striking in that

collective knowledge work proceeds not by interchange of statements alone; but most often

through statements followed by a question(s). Group 1 Facilitator posed more than 50% more

questions in both modules examined, than Group 2 Facilitator.

7.4.4.2.2 Results of analysis of complexity of discourse. Social network analysis of

complexity of clinical discourse was similar across all modules in that almost all student and

facilitator discourse was rated as complex or elaborated. Since all participants in this study are

family physicians, it is not surprising that all clinical discourse reflects a high level of

complexity, within notes, across all four modules scored.

7.4.4.2.3 Emergent themes/threads in knowledge work (beyond predefined learning

objectives). Emergent themes were identified in Group 1 and Group 2 discourse. Emergent

knowledge work can be characterized as adductive and abductive Knowledge Building, working

to improve knowledge so that it is understood more deeply and broadly. Group 1 and Group 2

identified different themes for knowledge work according to their needs and interests. Analogous

cases, “my experience,” “in my practice,” emotions, and pragmatics of practice (e.g., OHIP

billings). Personal knowledge lacks were identified, “need to know” questions were posted,

misconceptions corrected, and grey areas discussed that have no one answer (e.g., the art and the

science of palliative care practice).

7.4.4.2.4 Results of Knowledge Building indicators within social network discourse. All

Knowledge Building principles were evidenced in each module scored and most were found to

be “strong and consistent.” Exemplars of principles of community responsibility, democracy,

220

improvable ideas, epistemic agency demonstrated primarily through emergent themes/ideas, are

indicative of abductive/adductive Knowledge Building.

This comprehensive summary of results provides a big picture of the relationships across

findings. Results will be further elaborated in the next chapter, the Discussion chapter and study

conclusions, will also be framed by the three clusters of research subquestions: (a) traditional

outcome measures; (b) performance “over and above” traditional measures (beyond learning as

traditionally conceived and measured); and (c) questions regarding the sociocognitive dynamics

that enable work over and above traditional learning.

221

CHAPTER 8

DISCUSSION

8.1 Introduction

The chapter begins with a summary of the research, followed by a discussion of study results

based on relevant literature previously reviewed. Discussion of results will be organized

according to the three study clusters, related research questions, and findings.

8.2 Summary of Research

The current study was conducted in a continuing medical education End-of-Life Care Distance

Education course, for family physicians, from 2004 to 2009. A mixed methods case study

methodology was used to determine if socially mediated Knowledge Building improved

physicians’ knowledge, and if so, what social network structural relationships and sociocognitive

dynamics support knowledge improvement, democratization of knowledge, and meta-design of

Knowledge Building in continuing medical education.

Results from this multiyear study supports the use of a Knowledge Building approach in

continuing medical education, aimed at improving individual outcomes through community

opportunities for collective knowledge work, over and above traditional learning (Bereiter &

Scardamalia, 2003; Scardamalia & Bereiter, 2003a, 2005, 2006). Bereiter & Scardamalia (2007)

stated: “We do not suggest that ‘design mode’ should replace ‘belief mode.’ . . . Both are

important. . . . Knowledge creation depends on moving back and forth between belief and design

questions in ways that maintain progress in idea improvement” (p. 26).

The goal of this study was to show improvements according to traditional as well as

nontraditional measures—to show that there can be improvements, not tradeoffs, on both fronts.

The End-of-Life Care Distance Education program demonstrated consistently strong individual

learning and collective outcomes in the pilot and across 4-years. In the final year of the study,

2008/2009 social network structural measures were examined to describe the positive findings

and shed light the relationships across variables that support community Knowledge Building.

These relationships included high network density of build-on notes, well-distributed measures

centrality, and strong group coherence. Further in-depth examination of sociocognitive dynamics

showed shared facilitator/student core position and shared power of idea centrality. Within note

222

analysis clarified facilitator/student discourse structure of partner/expert and sociocognitive

dynamics of idea complexity and evidence of emergent themes for knowledge work, beyond the

learning objectives. Further content analysis demonstrated the presence of numerous Knowledge

Building indicators, such as community responsibility, idea improvement, epistemic agency, and

democratization of knowledge.

A summary of results is provided in Table 53. Discussion of results, based on the previous

literature review, will be similarly organized around the three study clusters: (a) traditional

outcome measures; (b) performance over and above traditional measures (beyond learning as

traditionally conceived and measured); and (c) sociocognitive dynamics that enable work over

and above traditional learning. Discussion of results of subquestions in this chapter, are used to

inform my answer to the research question in the Conclusions chapter.

Table 53

Clusters Summary of Research Questions and Results of Analyses

RESEARCH QUESTION

Does Knowledge Building improve physicians’ knowledge and understanding of palliative care in a Web-based

continuing medical education course; and if so, what social network structural relationships and sociocognitive

dynamics support knowledge improvement, and contribute to democratization and metadesign of Knowledge

Building in continuing medical education?

SUBQUESTIONS


Subquestion 1 Did students’ pain knowledge improve from pre to posttest and if so, is this improvement

significant and what is the effect size?

Data Analysis Results 2005-2009

2-tailed t-test on matched pre-/posttests Significant knowledge improvement (9% on

paired t-test = 5.34, p < 0.001)

Cohen’s d effect size Large effect (0.82)

Subquestion 2 Traditionally continuing medical education is a self-directed or didactic experience; what are

participants’ attitudes and opinions towards collaborative online knowledge building in the

End-of-Life Care Program?

Data Analysis Results 2005-2009

Likert scale and Yes/No questions in

online attitudes and opinions survey

Overall very positive, e.g. 86.7% rated their

online collaborative KB experience as above

average/excellent.

(table continues)

223


CLUSTER 2 PERFORMANCE MEASURES OVER AND ABOVE TRADITIONAL LEARNING

Subquestion 3 What are participants’ online activity and interactivity measures, by module, by group, by

year, in the 2005-2009 study (in each of the five modules, across two groups, and over four-

years)?

Data Analysis Results

5 online modules, by group, each year;

ATK tools in Knowledge Forum (KF)

2005-2009 overall high levels of read, write,

and build-on activity across all years.

Activity graphs and SNA of online

modules, over two groups using ATK tools

in KF

2008/2009 results demonstrate differences in

network density of build-ons between groups

and high facilitator participation


pretest to posttest, and is there a significant difference between groups?

Data Analysis Results 2008/2009

2-way ANOVA Gr. 1 and 2 pre-/posttests Difference over group was significant

F(1,16)=17.97, p<.001; difference between

groups was not; Gr.1 knowledge gains > Gr. 2


greater in Group 1 than in Group 2. What are the social network structural differences between

groups that supported increased knowledge improvement and how are these differences

related to centralization and/or democratization of participation and ideas?

Structural SNA Data Analysis (Group 1 and Group 2) Results 2008/2009

Network edges of build-ons and notes read

with/without facilitator = 20 analyses

Network links greater in Gr.1 than Gr. 2

Network density of build-ons and notes read

with/without facilitator = 20 analyses

Network density greater in Gr. 1 (0.278) than

Gr. 2 (0.182); difference in density for notes

built-on significantly stronger in Group 1 and

effect size very large (1.31); no significant

difference with/ without facilitator.

Eigenvector centrality of build-ons and

notes read with/without facilitator = 20

analyses

Centrality measures indicative of well

distributed groups in both Gr. 1 and Gr. 2; ); no

significant difference with/without facilitator.

Indegree/Outdegree centrality of build-ons

and notes read with/without facilitator = 20

analyses

Significant difference and large effect size

found for Gr. 1 and 2 with/without facilitator;

therefore, indicative of a high degree of

facilitator participation.

Cohesion index and cliques of build-ons

notes (including facilitator)

(measures: cohesion of ideas, no. of cliques,

no. of members, etc.)

Gr.1=27 cliques and Gr. 2=14; Gr. 1 facilitator

participated in all cliques; Gr. 2 facilitator only

participated in 8/14. Strong within group

cohesion and bridging between idea clusters

Correlation of social network centrality

variables (of combined groups) and pain

posttest knowledge score (Spearman’s rho)

Centrality correlations with posttest scores

were all positive.

224




Subquestion

6





Data Analyses (Group 1 and Group 2) Results 2008/09


(10 measures and mapped visualizations)

SNA of position/power/idea centrality and

relationship to knowledge improvement

Shared core position of power (often/but

not always with students with highest

pretest scores); students and facilitators

exhibit idea centrality

Identification of whose ideas are at the core, mid, or

periphery (facilitator and/or students; Aviv et al.,

2003)

Work in mid and peripheral fields by

greatest gainers (students with greatest

pre-/posttest knowledge gains); associated

with notions of identity.


(40% of the dataset was analyzed)

Facilitator stance Facilitator sociocognitive dynamics were

most often categorized as partner/expert

(Modified protocol of monitor, monitor/mentor,

mentor, participant, participant/expert, or expert

used based on Tabak & Baumgartner, 2004)


work?

Questions, as opposed to statements only,

dominated the discourse. Questions were

asked by students and facilitators.






Emergent themes were identified in the

discourse over and above learning

objectives; numerous emergent themes

were generated by Gr.1 (more than Gr. 2);

Themes broadened and deepened ideas, in

abductive and adduction knowledge work.



(Bordage, 1994; Bordage & Lemieux, 1991)

Discourse was categorized as

elaborated/compiled (as opposed to

reduced/dispersed)


Evidence of Knowledge Building principles scored

within notes (Use of scale by Sibbald, 2009 and

definitions by Scardamalia, 2002)

Evidence within and across notes of

Knowledge Building principles, such as

community responsibility, democratization

of knowledge, improvable ideas, epistemic

agency, intentionality to rise-above and

work with emergent ideas.

225

This case study was conducted in “Pasteur’s quadrant” defined as “use-inspired basic research,”

which lies at the intersection between pure basic research, aimed at improving understanding,

and purely applied research and development, aimed at improving technology (Stokes, 1997, p.

88). Outcomes of this study will be discussed in the next sections, as well as implications for

next-generation Knowledge Building and Knowledge Forum assessment tools.

8.3 Significance of Cluster 1 Research Results: Traditional Learning Outcomes

and the Relationship to Continuing Medical Education,

Traditionally Conceived

Continuing medical education, as traditionally conceived, has been harshly criticized as being

ineffective, as we have seen demonstrated by numerous papers in the literature review, and

particularly by Davis et al. (1999). In response, continuing medical education is currently being

reconceptualized and redefined. New strategic plans are emerging to address issues of

effectiveness and some, like the University of Toronto (2011) are advocating greater focus on

community and creative knowledge work, while others have redefined the term, to help reframe

objectives; for example, like Price, Havens, and Bell (2011) now refer to continuing medical

education as continuing professional development and improvement—a term that places the

focus beyond learning. Despite this, government regulatory bodies are slow to follow and

continue to defined continuing medical education in terms of individual knowledge improvement

and reward individual accomplish with learning activity credits, as opposed to performance

improvement credits (Dunikowski, 2011). Hence, the traditional measures of knowledge

improvement and program satisfaction, as used in the initial phases of this study, remain

important. Change is slow. As indicated by the results in this research cluster, positive

knowledge gains were demonstrated along with positive feedback on various socially mediated

aspects of Knowledge Building in the End-of-Life Care Distance Education Program across all

years.

Despite these positive outcomes measures, it is evident that to only measure individual outcomes

of knowledge improvement, in a course that focuses on collective improvement and community

knowledge work, only measures some of the outcomes - the traditional learning outcomes, and

not the community outcomes.

226

8.4 Significance of Cluster 2 Research Results: Social Network

Performance Measures Over and Above Learning

Knowledge Building in this continuing medical education course has been described as

successful in terms of knowledge gains demonstrated on pre-/post-test, positive attitudes

and opinions. Beyond those traditional measures, are measures of online activity and

interactivity of collective knowledge work, and most importantly in this study, social

network analysis measures that describe relationships of community engagement in terms of

structure, power, and sociocognitive dynamics of collective knowledge work. Across all 4-

years, the 2008/2009 cohort demonstrated the largest knowledge gains on pre-/posttest

scores and were therefore chosen for further analysis, to explore what made them

successful, not only in their individual knowledge improvement scores, but as a

collaborative Knowledge Building community.

Results of high online read/write/build-on activity and interactivity measures are common to

Knowledge Building research. However, those descriptions barely scratch the surface of the

intricate relationships revealed through social network structural and content analyses, as

found in this study. In the current study, almost all participants read notes, most contributed,

but only some “built-on.” Build-on notes were seen to be intentionally aimed at Knowledge

Building and became the central focus of this study. Strong relationships were demonstrated

between pain knowledge improvement and social network structural measures across groups

in 2008-09. When Group 1 and Group 2 were compared, relationship patterns indicate that

higher performance on Group 1 pre-/post knowledge tests were related to higher levels of

build-on notes in KF, higher number of network edges, significant difference in density of

build-on notes, lower coherence index and centrality measures, as well as, a greater number

of cliques and facilitator engagement, in an opportunistic and evenly distributed network.

The significance of these social network measures is that describe dimensions of collective

engagement and relationships between structures and how the community works with

knowledge and ideas. As we have seen, a paucity exists in the literature in regards to

sustained Knowledge Building studies of continuing medical education that describe online

communities using social network analysis. Punja’s research study (2007) only brief

examined Knowledge Building in this context and Mylopoulos’ study (2007) focused on

physicians in everyday practice. Neither of these studies used social network analysis as a

227

community measure. Both studies expressed concern about physician attitudes toward

creative knowledge work.

Community measures of improvement or success are not typically employed in continuing

medical education, or in education on the whole. However, with the new CanMEDS

competencies for physicians (Frank, 2005), of collaborator and scholar, there may be reason

to believe that measure of collective knowledge work using tools such social network

analysis may become necessary. Similarly, as seen in the literature, recent

recommendations, like the University of Toronto, Faculty of Medicine, Office of CEPD’s

strategic plan that aim to create more opportunities for collective work and the use of

technology, will likely also need new measure to support collective performance assessment

(University of Toronto, 2011).

The literature reviewed in regards to this cluster provides perspectives on Knowledge

Building theory, Knowledge Forum analytical tools, and social network analyses, with

relevant research studies drawn from the domains of education, the social sciences, and

medicine. This study makes a unique contribution to the research literature across domains

of Knowledge Building and social network analysis and continuing medical education.

Emergence is a central idea and indicator of Knowledge Building, and one that distinguishes it

from learning (Bereiter, 2002a; Bereiter & Scardamalia, 1996). Emergent ideas and design mode

work in the End-of-Life Care Distance Education Program can be characterized as authentic,

working with real ideas and real problems (Bereiter & Scardamalia, 2003; Scardamalia, 2002).

Although the cases created are based on real world cases and the objectives identified relate to

real-world problems they are considered 1-step removed since they are predetermined and

instructor-developed, as opposed participant-developed, emergent ideas. However, as

demonstrated in this study one groups’ knowledge work was almost evenly split between

working with predetermined objectives and community emergent ideas.

Traditional learning environments are often far removed from real-world practice. This

disconnect in continuing medical education is one reason why efforts have shown little effect in

changing physician performance (Davis et al., 1999). It can be argued that with opportunities for

authentic discourse there is less of a leap from the world of education to practice—and less

228

concern for current remedies for far-removed endeavours. The area knowledge translation was

specifically created to address this issue (Graham et al., 2006; Davis, Evans et al., 2003).

In “Weaving the Web,” creator of the World Wide Web, Tim Berners-Lee (1999), reflected:

I have a dream for the Web . . . and it has two parts.

In the first part, the Web becomes a much more powerful means for collaboration

between people. I have always imaged the information spaces as something to which

everyone has immediate and intuitive access, and not just to browse, but to create. . . .

In the second part of the dream, collaborations extend to computer. Machines become

capable of analyzing all the data on the Web—content, links, and transactions between

people and computers. A “Semantic Web,” which should make this possible, has yet to

emerge… (pp. 157–158)

Teplovs and Scardamalia’s (2007) recent work on visualization of assessments is evidence that

we are moving into an era of the Semantic Web; again tools are becoming necessary for analysis

of collective online work. Social network analysis tools have been recently integrated in

Knowledge Forum for use not only by researchers but by participants too. Future studies will

explore the use of these social network tools by participants involved in collective Knowledge

Building to monitor and assess their own processes and performance in relationship to others in

the network.

In the End-of-Life Care Distance Education study social network patterns across time (five

months) and databases (Group 1 and Group 2) demonstrated differences in structure and

distribution, both of knowledge/ideas, and participation. Each of the 10 network modules

demonstrated its, own unique patterns and relationships. Each had its own unique set of

characteristics.

Importantly, parallel and overlapping patterns of social network structural results became

evident. This resonance of findings created relationships across measures and attributes to

describe the network. For example, Network density of ideas proved to be very informative and

produced positive parallel findings of distributed centrality measures and cohesion. Hence these

datasets capture, represent, and help us visualize how social networks are structured and how

ideas are connected and worked with.

An interesting parallel resides within Knowledge Forum technology itself; it provides views of

notes and note connections in each view, or problem space. In Knowledge Forum, a threaded

discourse view can be almost instantaneously changed to a network view, which not only

demonstrates participation connections, who built on whose notes, but also relationships between

229

ideas, and clusters of ideas. Hewitt (2001) in a 1997 presentation published in 2001 as “Beyond

Threaded Discourse,” used Knowledge Forum technology to visually represent ideas as networks

(as opposed to threads), for analysis in his study. Knowledge Forum technology enabled easy

and transparent visual identification of idea clusters. Hewitt identified the importance of this type

of technological support in the early phase of development of Knowledge Forum, when the

technology was still WebCSILE. In his paper he stated:

First, enhancements are needed to facilitate idea linking and connectivity. Second,

collaborative group processes need to become more visible. Specifically, a set of tools are

required that permit individuals to better monitor communal knowledge-building activity

and their own participation in that activity. (Hewitt, 2001, p. 207)

Over 10 years later Hewitt’s vision became a reality—realized through the creation of social

network work and other important tools, for participant and researcher use, in Knowledge

Forum.

Numerous social network analyses were conducted in the current study and tools were required

beyond those available in Knowledge Forum. Although network density was found to be an

extremely informative measure in this study, it is currently the only tool available in Knowledge

Forum. Based on the experience of this study, I recommend expanding the suite of social

network tools to include measures of network centrality, cohesion, and cliques.

8.5 Significance of Cluster 3 Research Results: Sociocognitive Dynamics

That Enable Work Over and Above Learning

Discussion of results in this cluster will begin with an overview of facilitator/participant

structures and then focus on principles and sociocognitive dynamic indicators of Knowledge

Building (Scardamalia, 2002), associated network measures, and content analysis results.

8.5.1 Facilitator/Participant Sociocognitive Dynamics

Student Knowledge Building opportunistic networks characteristically do not have a

prominently engaged facilitator; however, in this continuing medical education course one

facilitator was found to be extremely engaged, involved in all cliques; she assumed the role

of “partner or partner/expert” in the discourse, without dominating or controlling the

discourse, enabling democracy and emergence.

230

Although facilitators were persistently positioned in the central power position in the

network, that position was almost always shared with students. Surprisingly, greatest

knowledge “gainers” on pre-/posttests, were not always found working at the core, but

sometimes at the mid- and periphery of the problem space. Similarly, contrary to

expectations, it was posited that students with the highest scores on pain pre-test would

assume the core position, sharing this position with the facilitator, which in fact was the

case some of the time, but not always. It appears that the best and the brightest do not

always want to be in the position of power and authority and teach others; some assume

positions at the periphery and are relatively silent.

Social network centrality maps and measures of power of ideas and network position proved

to be an interesting and informative way to assess relationships within the community. This

is a promising avenue of further Knowledge Building research and builds-on previous

research by Aviv and colleagues (2003) on cohesion, role, and power; as well as Cornelius

and Herrenkohl paper on how power shapes relationships between students and ideas

(2004); and Tabak and Baumgartner’s (2004) work on teacher/student structures; and

Ligorio’s (2009, 2010) research on identity.

8.5.2 Belief- and Design-Mode Knowledge Work

Family physicians register in End-of-Life Care Distance Education Program because they are

interested in improving their knowledge and learning “facts” about palliative care. Needless to

say, there are many important facts to be learned, e.g. how to titrate pain medications. However,

what we saw in the discourse was not only the mathematics of pain medication titration, but

ideas around treatment. Since each patient reacts to different pain medications differently, good

pain management, like many aspects of medicine, is both a science and an art. Hence,

opportunities, beyond learning, for design mode work are required.

Design mode work also became evident in the real-world clinical cases participants posted for

discussion. Use of analogies (Thagard, 1997; Holyoak & Thagard, 1995; Glick & Holyoak,

1980) and abductive reasoning (Thagard & Shelley, 1997) in this way is important to create

deeper understanding and personal meaning (Bruner, 1990).

Thagard posits that “We can judge that a scientific theory is progressively approximating the

truth if it is increasing its explanatory coherence is two key respects: broadening by explaining

231

more phenomena, and deepening by investigations layers of mechanisms” (Thagard, 2007, p.

28). Broadening and deepening ideas, through abductive and adductive knowledge work,

respectively, has important possibilities for Knowledge Building and provides evidence of the

sociocognitive dynamics of the principle of improvable ideas, described as:

Improvable ideas. Sociocognitive dynamics. All ideas are treatable as improvable.

Participants work continuously to improve the quality, coherence, and utility of ideas. For

such work to prosper, the culture must be one of psychological safety, so that people feel

safe in taking risks—revealing ignorance, voicing half-baked notions, giving and

receiving criticism. (Scardamalia, 2002, p. 78)

Further research is needed on the use of analogies and abductive and adductive reasoning, and

implications for coherence of ideas to improve ideas in collective Knowledge Building.

8.5.3 Knowledge Building Principles and Metadesign Concepts

As we have seen in the End-of-Life Care Distance Education study opportunities for

collaborative participation are fundamental to design mode knowledge work. In an emergent

problem space continuing medical education participants have opportunities to be involved at the

metadesign level of curriculum—normally reserved for instructors, teachers, experts, etc. The

Knowledge Building principles of rise-above and epistemic agency drive this process forward.

Scardamalia (2002) defines these principles below.

Rise above. Sociocognitive dynamics. Creative knowledge building entails working

toward more inclusive principles and higher-level formulations of problems. It means

learning to work with diversity, complexity, and messiness, and out of that achieve new

syntheses. By moving to higher planes of understanding, knowledge builders transcend

trivialities and oversimplifications and move beyond current best practices (Scardamalia,

2002, p. 79).

Epistemic agency. Sociocognitive dynamics. Participants recognize both a personal and a

collective responsibility for success of knowledge building efforts. Individually, they set

forth their ideas and negotiate a fit between personal ideas and ideas of others, using

contrasts to spark and sustain knowledge advancement rather than depending on others to

chart that course for them. Collectively they deal with problems of goals, motivation,

evaluation, and long-range planning that are normally left to teachers or managers

(Scardamalia, 2002, p. 79).

In the context of continuing medical education, participant identification of ideas that require

work and improvement in their specific practice context is particularly important, for authenticity

and relevance. Few continuing medical education curricula enable participant identification of

ideas to emerge, let alone “rise above” the prescribed curriculum. Scardamalia’s concept of rise-

232

above specifically addresses the problematic notion of regarding best practices as an end-state

object—as dogma—that cannot be revised.

Epistemic agency “refers to the amount of individual or collective control people have over the

whole range of components of knowledge building—goals, strategies, resources, evaluation of

results, and so on” (Scardamalia and Bereiter, 2006, p. 106). This description is the opposite of

typical, “top-down” designed, structured, didactic, continuing medical education where students

are given little to no control over these components, and definitive hierarchies of power are

evident. Epistemic agency relates to the concept of metadesign, where one is able to take control,

or collaboratively participate in all high level planning, processes, creation and design.

8.5.3 Collective Responsibility and Democratizing Knowledge

Scardamalia & Bereiter (2006) posit that one of the key limitations to Knowledge Building is

when a community is structured with the teacher as the hub of information. Altering that

structure and flow of information from teacher to students, which is classically hierarchical and

didactic, is essential.

As evidenced in the study herein, shifts in power structure, flow of information, and epistemic

agency can be demonstrated by change in network patterns over time. In the End-of-Life Care

Distance Education study we demonstrated the centralized but shared tendency of the facilitator

with high levels of community engagement and demonstration of community responsibility for

emergent ideas and design-mode knowledge work. Scardamalia’s (2002) description of the

sociocognitive dynamics of democratizing knowledge coupled with the principle of community

knowledge/collective responsibility, key principles in this study are defined below.

Community knowledge, collective responsibility. Sociocognitive dynamics.

Contributions to shared, top-level goals of the organization are prized and rewarded as

much as individual achievements. Team members produce ideas of value to others and

share responsibility for the overall advancement of knowledge in the community

(Scardamalia, 2002, p. 80)

Democratizing knowledge. Sociocognitive dynamics. All participants are legitimate

contributors to the shared goals of the community; all take pride in knowledge advances

achieved by the group. The diversity and divisional differences represented in any

organization do not lead to separations along knowledge have/have-not or innovator/non-

innovator lines. All are empowered to engage in knowledge innovation.

233

Technological dynamics. These is a way into the central knowledge space for all

participants; monitoring tools assess evenness of contributions and other indicators of the

extent to which all members do their part in a join enterprise (Scardamalia, 2002, p. 80).

Technological dynamics related to sociocognitive dynamics of democratizing knowledge are also

presented for this principle. As indicated by this study the use of social network analyses tools in

Knowledge Forum and other software, proved to be very informative and thus should be

included in the next revision of Knowledge Building indicators.

The Knowledge Building principle of democratizing knowledge can be explored in new ways as

a result of the embedded Knowledge Forum visualization tools. Distributions of power across

ideas and participants can be determined and visually demonstrated as autocratic or democratic.

An autocratic environment, that is characterized as teacher-centred and by knowledge telling,

transmission, and the unidirectional flow of ideas can now be seen, like a snap shot of a point in

time. Similarly, all kind of democratic distributions can be visualized across participates. The

Knowledge Forum visualization tools make these dynamics “tacit” (Polyani, 1987).

The link between democracy and education is not new, and in fact John Dewey wrote an entire

book on its various dimensions, published in 1916. Dewey conceives of what he calls the

“democratic criterion” in education as “to imply the ideal of a continuous reconstruction or

reorganizing of experience, of such a nature as to increase its recognized meaning or social

content, and as to increase the capacity of individuals to act as directive guardians of this

reorganization” (Dewey, 1916, p. 188). Dewey’s “democratic criterion” is reflected in

Scardamalia and Bereiter’s principle of democratizing knowledge.

Fischer (2009a) referred to “democratizing design” as fostering cultures of participation in next-

generation computer-supported collaborative learning systems and explores authoritative and

democratic models. He indicated that the disadvantage of the authoritative model is that it limits

the number of voices heard; the controlling mechanisms suppress broad participation from

different constituencies. Fischer referred to “democratized design cultures” and noted that the

disadvantages of this model were its potential for reducing trust in and reliability of content, and

explosion of information. He indicated that cultures supported by the authoritative model

encourage consumption of polished, finished products. In contrast, emergent democratized

design cultures are characterized by a wider, richer set of cultural forms and practices that

require new forms of computer supported collaborative learning.

234

Democratized design cultures, such as those engendered by knowledge building, attempt to

articulate ideas about education and the trajectory of lifelong learning. Bereiter (2002a) argued

for liberal education to support a knowledge society, and by liberal he means essentially

enculturation. Bereiter (2002a) proposed that “school be a productive part of that society, a

workshop for the generation of knowledge” (p. 12) and that the process of Knowledge Building

continues beyond school, into work and throughout life. This is a fitting aim for continuing

medical education too—that continuing medical education “be a productive part of that society, a

workshop for the generation of knowledge” (Bereiter, 2002a, p. 12)

The proximity of politics and education, and democracy and knowledge, and ideas on

enculturation, brings us to the question of “responsibility,” whether it be civic or professional.

Framed by the context of this study, we can ask: What is a physician’s professional responsibility

to improve knowledge? The answer is disappointing; it seems not much (Mylopolous, 2007).

Other results indicated that physicians do feel professionally responsible to improve quality of

care (Brennan, 2002). However, improving “quality of care” is different—it is much more

limited than taking responsibility for improving knowledge. Physicians may feel drawn to

improve quality of care because they are engaged in care but often remain removed from

knowledge improvement—and this may be due to the way in which continuing medical

education and professional development are current conceived. The democratization of

continuing medical education aims to turn over individual and collective responsibility to

improve knowledge, ideas, and practice—by making continual professional development

personally meaningful and relevant—through participatory engagement and metadesign.

Democracy, shifts in power, flow of ideas, and epistemic agency are not elements usually

assessed in educational environments. However, it is noteworthy that these design mode types of

assessment are newly informative and complement traditional belief mode knowledge

assessments and indicators of change. Changes in democratic relationships captured by new

Knowledge Forum assessments and positive improvement in knowledge are highlighted herein,

in the End-of-Life Care Distance Education study. These exciting new ways of seeing,

demonstrating, and evaluating change has important implications for future research and enables

us to ask new kinds of questions. We will need to ask and investigate questions like: Do the tools

correlate with design mode Knowledge Building and ultimately indicate network areas of “Big

Ideas,” hot spots of Knowledge Building and innovation (Scardamalia & Bereiter, 2005, p. 28)?

235

Tools to analyze distribution and symmetry of power would be helpful for researchers and

participants working in Knowledge Forum.

Michel Foucault (1994) told us that power distributions become evident within discourse.

Foucault was interested in macro sociocultural relationships that are useful for our framing vis à

vis Knowledge Building. Configurations of power, knowledge, and truth constitute discourse for

him. “One of the most productive ways of thinking about discourse is not as a group of signs or a

stretch of text, but as ‘practices that systematically form the objects of which they speak’”

(Foucault, 1972, p. 49). In this sense, a discourse is something which produces something else

(Mills, 1997). Foucault does not consider discourse as something that exists in and of itself that

can be analyzed in isolation, acontextually. A discursive structure can be demonstrated through

“systematicity of ideas,” ways of thinking and behaving in a context, and the “effects” of those

ways of thinking and behaving (Mills, 1997). The effects of discourse important to consider are

the elements of truth, power and knowledge. Truth, interests Foucault, from the perspective of

production and mechanics—how social practices regard questions of truth and authority. Power,

according to Foucault is dispersed throughout social relations and produces possible forms of

behaviour as well as restricts.

Knowledge Building in Knowledge Forum enables us to look at power relations in association

with examination of “truth” and idea advancement. Although the End-of-Life Care Distance

Education study did not examine discourse through this type of text analysis, the framing of

discourse as an object, with characteristics of power relations, effects, and as an evolving system

over time is implicit in the analyses work done herein. Foucauldian ideology may provide an

important framework for future research at the grain of semantics.

Roger Saljo (2003) in a keynote address titled: “Representational Tools and the Transformation

of Learning,” makes the point that new technologies do not enhance learning; however, if they

are powerful enough, they transform basic features of how people communicate knowledge, how

knowledge is organized, and how it is represented. Knowledge Building representations, of ideas

in discussion notes, interaction patterns in databases, and social network assessments for research

and concurrent feedback, have the potential to transform current conceptualizations of continuing

medical education.

236

8.6 Summary

This study advocates based on the results presented herein, pushing the current agenda of reform

for continuing medical education even further forward—to go “beyond best practices”

(Scardamalia, 2002) - to reframe continuing medical education within a Knowledge Building

approach, for intentional cognitive collaborative knowledge work—for meaningful and relevant

lifelong learning and sustained improvement of ideas, expertise, and patient care.

This study provides strong evidence for change and exemplars of social network

collaborative participation, emergent ideas, and metadesign of curriculum. These aspects

provide a community model for collective Knowledge Building to address current

deficiencies in continuing medical education. This is the first study to address

sociocognitive dynamics of a Knowledge Building continuing medical education online

community. The conclusions of this study are presented next.

237

CHAPTER 9

CONCLUSIONS

9.1 Introduction

This chapter includes a series of conclusions summarizing research findings, as well as strength

and limitations of research, and implications and recommendations for future research.

9.2 Research Questions and Answers

In this study I asked:


palliative care in a Web-based continuing medical education course, over a five-year

period; and if so, what social network structural relationships and sociocognitive

dynamics support knowledge improvement, and contribute to democratization and

metadesign of Knowledge Building in continuing medical education?

The course on which this research is based is ongoing; the research covers a five-year period,

2004 to 2009, and presents results from a series of investigations based on notes entered into the

course environment, Knowledge Forum, by physicians. Results continue to inform the design of

this course.

Results of this study demonstrate that physicians’ knowledge and understanding of palliative

care improved while working collectively online. Perhaps that could be achieved in any

effectively designed course. In the current context these results were important because they

provided the backdrop for broader goals; namely, to demonstrate advances beyond those that can

be achieved in well-designed courses. Thus the finding that physicians’ knowledge and

understanding of palliative care improved while working collectively online provides the basic

finding. Detailed examination of the social network structural relationships and sociocognitive

dynamics that enabled individual achievement also enabled collective knowledge improvement,

as elaborated below.

This study demonstrated how physicians can create a culture of participation and work

collaboratively to improve knowledge and understanding, and in doing so address not only

238

predefined learning objectives, but work over and above through emergent ideas, to contribute to

the design of the curriculum. This is what I have called democratization of knowledge and

metadesign in this case study. These findings reflect Knowledge Building principles of

community responsibility, epistemic agency, idea improvement, and others. Through these meta

processes physicians’ emergent ideas became a strong focus of discourse; their agenda was

addressed, alongside that of the predetermined learning objectives from the content experts. The

sociocognitive dynamics of Knowledge Building, over and above learning, provide a unique

example in this case study of the potentiality of Knowledge Building democratization and

metadesign of continuing medical education.

To summarize, in this study, traditional pre-/posttest learning measures across 4-years showed

significant gains (9% on paired t-test = 5.34, p < 0.001) and large effect size (0.82). Social

network analysis of ten 2008/2009 modules showed significant difference in density of build-on

notes across groups. Additional results demonstrated a relationship between high knowledge

gains and social network measures of centrality/distribution and cohesion. Correlation of posttest

scores with centrality variables were all positive. Position/power analyses highlighted core-

periphery sociocognitive dynamics between the facilitator and students. Facilitators most often

evoked partner/expert relationships. Questions rather than statements dominated the discourse;

discourse complexity was elaborated/compiled as opposed to reduced/dispersed. Themes beyond

pre-defined learning objectives emerged and Knowledge Building principles of community

responsibility, idea improvability, and democratization of knowledge were evident.

Overall, results of the current study showed positive effects of collective Knowledge Building, as

demonstrated by the social network structural relationships and sociocognitive dynamics that

support a culture of participation, democratization, and metadesign of continuing medical

education, over and above individual accomplishments. The potential of collective Knowledge

Building and design-mode work for continuing medical education is evident, with individual

learning representing an important by-product. As demonstrated in this study, there were no

discernible decrements in individual performance, suggesting significant advantages rather than

tradeoffs from engagement in collective knowledge building.

239

9.3 Strengths of This Research Study

Strengths of the research rest with theory-based design and research to examine a novel approach

to collective Knowledge Building as defined by Scardamalia and Bereiter (2003a) and to explore

sociocognitive dynamics of underlying principles (Scardamalia, 2002). Physician participation in

the End-of-Life Care Distance Education Program over a five-year period, from 2004 to 2009,

provided a robust dataset for the research. To realize these strengths it was necessary to design

an educational program with sufficient power to enable the advances represented in the research.

Thus the course itself provides an important artifact—one that is being continually improved and

implemented in the field of continuing medical education.

The case study research methods implemented allowed for naturalistic examination of the

ongoing discourse of participants, as these discourses were captured in the Knowledge Forum

online environment. All participants agreed to participate in the research component, with

assurances of anonymity (all real names were replaced with pseudonyms—even those in the

exemplar Knowledge Forum notes). This study provided us with an opportunity for authentic,

naturalistic, and holistic research to examine complexities and nuances in situ, as physicians

engaged in learning and knowledge building.

An additional strength was the use of social network analysis to assess and demonstrate

collective outcomes and relationships. Various methods were used to assess different dimensions

and a series of parallel findings emerged, with one supporting the other, enabling interdigitation

of methods and outcomes. For example, this study clarified social network structural dimensions

of network density, centrality, and coherence. Strength of participation, distribution, and

democratization between facilitator and students was further supported by findings of social

network position and power analysis.

Content analyses of notes were used to enrich other analyses and to advance our understanding

of knowledge cocreation and metadesign, facilitator/student sociocognitive dynamics, and

facilitator/participant structures underlying high level facilitator engagement. These analyses

were aimed at a meta level, in the sense that they were designed to enrich understanding of

sociocognitive dynamics, rather than strictly the level of the semantics within the discourse

(Hodges et al., 2008). The strength of results related to sociocognitive dynamics rests ultimately

240

with descriptions of social network relationships that scaffold collective knowledge building, and

the individual measures of learning, as by-product of community achievements.

9.4 Limitations

This study was limited by the number of participants working online each year. Annual and four-

year cumulative 2005–2009 analyses were employed on pre-/posttest and survey results to enable

tests of significance. The sample population was not randomized and/or selected according to

any criteria; all participants, who signed up for the course, were included in this study, if they

provided their consent.

The study focused on the strongest dataset—the groups with strongest outcomes. Thus the

research did not examine weaker groups, ineffective social network structures, or comparisons

between weak and strong networks across years. However, now that patterns associated with

strong outcomes have been identified this next stage in the research program will be greatly

facilitated. In addition, build-on notes were specifically chosen as the only focus of study for

deeper social network analysis of position and power, as build-on notes, by definition, show

linked clusters and they provide a clear basis for examining sociocognitive dynamics. Other

parameters such as note contribution or reading measures will be important to study in the future.

Findings may also be limited by the social network analysis tools used, preset parameters, and

assessments they are programmed to conduct. By the very definition of the course design and

methodology employed, this multiyear case study may not be generalizable to other contexts;

nonetheless, to the extent possible and within the scope of this investigation, mixed methods and

multiple measures were used to demonstrate consistency of findings and reliability of results.

9.5 Future Research

Additional research is needed for more detailed accounts of linkage between content analysis and

social network structural analyses. And as suggested from the account of above, future research

needs to address limitations in the current generation of automated social network and semantic

analysis tools. These tools are developing rapidly and show great potential for advancing

concurrent and embedded assessment in online environments (Lax et al., 2010). As methods are

refined for research they will open up much greater potential for providing more effective

feedback to the formation of knowledge advancing teams.

241

To measure individual learning in a knowledge building community is to measure only one facet

of knowledge work. This research shows that measures of community work and sociocognitive

engagement provide a more systemic approach focused less on patterns of difference among

individuals and more on the differences of patterns of relations among sociocognitive factors

(Salomon, 1994a, 1994b).

Current impoverishment of continuing medical education opportunities for collaborative

engagement and design mode innovation leaves ample room for improvement. Positive outcomes

of this study provide strong incentives for adoption of a knowledge building approach for next-

generation initiatives.

9.6 Final Remarks

Scardamalia and Bereiter (2010) argued that the emerging knowledge age poses an educational

challenge that can best be met by adopting a knowledge building approach and a key feature of

that approach is “democratization of innovation capacity” (p. 11). Unlike research scientists,

most physicians do not see themselves as a community of knowledge builders responsible for

idea advancement and innovation (Mylopoulos, 2007). As this study indicates when physicians

are engaged in a culture of participation and provided with opportunities for sociocognitive

work, they can take responsibility for idea improvement through creative, design mode work,

and work constructively as a Knowledge Building community.

This is the first extensive, multiyear study of Knowledge Building in continuing medical

education that focuses on understanding social network structural relationships and

sociocognitive dynamics, over and above learning. Positive outcomes of this study may inspire

curriculum designers and participants, themselves, to consider educational models that advance

beyond individualistic, teacher-centric learning, toward emergent, design mode dynamics in

environments that offer opportunities to boost collective capacity for change and innovation and

work with emergent ideas.

Adventures with ideas located in one’s head do not provide much opportunity for expansive

knowledge work (Whitehead, 1933, p. 278). When ideas are democratized, located at-the-centre,

objectified in a public space, opportunities for sociocognitive improvement and knowledge

innovation become possible (Bereiter, 2002; Scardamalia, 1999, 2002, 2003a, 2004b;

Scardamalia & Bereiter, 2003a, 2003b). The democratization of continuing medical education

242

may enable individuals to take collective responsibility for going beyond best practice and

evidence-based learning.

Continuing medical education defined as learning is necessary, but not sufficient, as

demonstrated in this study. Working at the edge does not come easy; it requires epistemic agency

and intentionality aimed at improving ideas and sustaining innovation. A new model of design-

mode education may engender a change in attitude and culture, from one that typically values

individual accomplishment to one that values collective achievement—a community that values

knowledge creation as well as knowledge acquisition.

Continuing medical education has been harshly criticized as ineffective; current evaluation and

reconceptualization has led to envisioning the potential of what might be. The importance of

reenvisioning the past to inform the future is captured in Whitehead’s (1933) words. He stated:

A race preserves its vigour so long as it harbours a real contrast between what has been

and what may be; and so long as it is nerved by the vigour to adventure beyond the

safeties of the past. Without adventure civilization is in full decay.

It is for this reason that the definition of culture as the knowledge of the best that has

been said and done, is so dangerous by reason of its omission. It omits the great fact that

in their day the great achievements of the past were the adventures of the past.

. . . A living civilization requires learning; but it lies beyond it. (Whitehead, 1933, pp.

278–279)

Whitehead’s notion of the potential of adventures with ideas to impact civilization is not unlike

Scardamalia & Bereiter’s, on the value of knowledge building to potentially effect change and

contribute to the advancement of knowledge to impact real world outcomes. Scardamalia and

Bereiter (2010) state: “Continual idea improvement is the hallmark of a progressive society, and

the present age calls for this responsibility to be much more widely distributed than before” (p.

12). The results and implications of this study, coupled with identified need for change in

continuing medical education, demonstrate the necessity for ongoing research-based innovation.

243

REFERENCES

Abrahamson, S., Baron, J., Elstein, A. S., Hammond, W. P. , Holzman, G. B. Marlow, B., . . .

Schulkin, J. (1999). Continuing medical education for life: Eight principles. Academic

Medicine, 74(12), 1288–1294. Retrieved from http://journals.lwww.com

/academicmedicine

Accreditation Council for Continuing Medical Education (ACCME©

). (n.d.). A directed self-

assessment for readiness to meet the ACCME’s updated accreditation criteria. Retrieved

from http://www.pamedsoc.org/MainMenuCategories/CMEAccreditation

/Organizations/Tools/Readiness.pdf

Akin, O. (2001). “Simon says”: Design is representation. Unpublished manuscript, School of

Architecture, Carnegie Mellon University, Pittsburgh, PA. Retrieved from http://www.

andrew.cmu.edu/user/oa04/Papers/AradSimon.pdf

American Medical Association (AMA). (2010). The physician’s recognition award and credit

system: Information for accredited providers and physicians. Retrieved from

http://www.ama-assn.org/resources/doc/cme/pra-booklet.pdf

Anderson, L., & Krathwohl, D. A. (2001). Taxonomy for learning, teaching and assessing: A

revision of Bloom’s taxonomy of educational objectives. New York, NY: Longman.

Appelbaum, P. (2002). Here comes lifelong learning. Psychiatric News, 37(19), 3–38.

Association of American Medical Colleges. (2004, July). Educating doctors to provide high

quality medical care. A vision for medical education in the United States (Report of the

ad hoc committee of deans). Washington, DC: Author.

Aviv, R., Erlich, Z., Ravid, G., & Geva, A. (2003). Network analysis of knowledge construction

in asynchronous learning networks. The Journal of Asynchronous Learning Networks,

7(3), 1–23. Retrieved from http://sloanconsortium.org

Barrows, H. S., & Tamblyn, R. M. (1980). Problem-based learning: An approach to medical

education. New York, NY: Springer-Verlag.

Bennett, N. L., Davis, D. A. Easterling, W. E. Jr., Friedmann, Green, P., Koeppen, B. M., . . .

Waxman, H. S. (2000). Continuing medical education: A new vision of the professional

development of physicians. Academic Medicine, 75(12), 1167–1172. Retrieved from

http://journals.lww.com/academicmedicine

Bereiter, C. (2000). Drilling down and building up. The Journal of the Learning Sciences, 9(2),

205–208. doi:10.1207/s15327809jls0902_4

Bereiter, C. (2002a). Artifacts, canons, and the progress of pedagogy: A response to contributors.

In B. Smith (Ed.), Liberal education in a knowledge society (pp. 223–244). Chicago, IL:

Open Court.

Bereiter, C. (2002b). Design research for sustained innovation. Cognitive Studies, Bulletin of the

Japanese Cognitive Science Society, 9(3), 321–327.

244

Bereiter, C. (2002c). Education and mind in the knowledge age. Mahwah, NJ: Lawrence

Erlbaum Associates.

Bereiter, C., Burtis, J., & Scardamalia, M. (1994). Problem-centered knowledge: Education

beyond content and process. Toronto, ON: OISE/UT.

Bereiter, C., & Scardamalia, M. (1989). Intentional learning as a goal of instruction. In L. B.

Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp.

361–392). Hillsdale, NJ: Lawrence Erlbaum Associates.

Bereiter, C., & Scardamalia, M. (1990). An architecture for collaborative knowledge building. In

E. De Corte, M. Linn, H. Mandl, & L. Verschaffel (Eds.), Computer-based learning

environments and problem solving (pp. 41–66). Berlin, Germany: Springer-Verlag.

Bereiter, C., & Scardamalia, M. (1993). Surpassing ourselves: An inquiry into the nature and

implications of expertise. Peru, IL: Open Court.

Bereiter, C., & Scardamalia, M. (1996). Rethinking learning. In D. Olson and N. Torrance

(Eds.), The handbook of education and human development: New models of learning,

teaching, and schooling (pp. 485–513). Malden, MA: Blackwell.

Bereiter, C., & Scardamalia, M. (1999). Beyond Bloom’s taxonomy: Rethinking knowledge for

the knowledge age. In M. Fullan (Ed.), The international handbook of educational

change. Dordrecht, the Netherlands: Kluwer.

Bereiter, C., & Scardamalia, M. (2003). Learning to work creatively with knowledge. In E. De

Corte, L. Verschaffel, N. Entwistle, & J. van Merriënboer (Eds.), Powerful learning

environments: Unravelling basic components and dimensions (pp. 55–68). Advances in

learning and instruction series. Oxford, UK: Elsevier Science.

Bereiter, C., & Scardamalia, M. (2007, January). Toward research-based innovation. In Centre

for Educational Research and Innovation (Ed.), Innovating to learn, learning to innovate

(pp. 67–91). Paris, France: OECD.

Berners-Lee, T., with Fischetti, M. (1999). Weaving the web: The original design and ultimate

destiny of the World Wide Web by its inventor. San Francisco, CA: Harper.

Bielaczyc, K. (2006). Designing social infrastructure: Critical issues in creating learning

environments with technology. Journal of the Learning Sciences, 15(3), 301–329.

doi:10.1207/s15327809jls1503_1

Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy

of educational objectives: The classification of educational goals. Handbook 1: Cognitive

domain. New York, NY: Longmans, Green.

Bordage, G. (1994). Elaborated knowledge: A key to successful diagnostic thinking. Academic

Medicine, 69(11), 883–885. Retrieved from http://journals.lww.com/academicmedicine

Bordage, G., Connell, K. J., Chang, R. W., Gecht, M. R., & Sinacore, J. M. (1997). Assessing the

semantic content of clinical case presentations: Studies of reliability and concurrent

245

validity [Supplement]. Academic Medicine, 72(10), S37–S39. Retrieved from


Bordage, G., & Lemieux, M. (1991).Semantic structures and diagnostic thinking of experts and

novices [Supplement]. Academic Medicine, 66(9), S70–S72. Retrieved from


Bradley, E. H., Cramer, L. D., Bogardus , S. T., Kasl, S. V., Johnson-Hurzeler, R., & Horwitz, S.

M. (2004, April). Physicians’ rating of their knowledge, attitudes and end-of-life-care

practices. Academic Medicine, 77(4), 305–311.

Brennan, T. A. (2002). Physicians’ professional responsibility to improve the quality of care.

Academic Medicine, 77(10), 973–980. Retrieved from http://journals.lww.com

/academicmedicine

Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating

complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2),

141–178. doi:10.1207/s15327809jls0202_2

Brown, A. L., & Palincsar, A. S. (1989). Guided, cooperative learning and individual knowledge

acquisition. In L. B. Resnick (Ed.), Knowing, learning and instruction: Essays in honor

of Robert Glaser (pp. 393–451). Hillsdale, NJ: Lawrence Erlbaum Associates.

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning.

Educational Researcher, 18, 32–42. doi:10.3102/0013189X018001032

Brown, J. S., & Duguid, P. (2000). The social life of information. Boston, MA: Harvard Business

School Press.

Bruner, J. (1990). Acts of meaning. Cambridge, MA: Harvard University Press.

Burt, R. S. (2004). Structural holes and good ideas. The American Journal of Sociology, 110(2),

349–399. doi:10.1086/421787

Burtis, J. (2001, September 28). Analytic toolkit for Knowledge Forum®

. Retrieved from

http://kf.oise.utoronto.ca/atk/cgis/atkdoc.html

Burtis, J., Chan, C., Hewitt, J., Scardamalia, M., & Bereiter, C. (1993, April). Misconceptions in

a student-guided research project. Poster session presented at the annual meeting of the

American Educational Research Association, Atlanta, GA.

Campbell, C. M., & Parboosingh, I. J. (2011). The evolution of continuing professional

development at the Royal College of Physicians and Surgeons of Canada: Setting

standards for Canadian specialists. In D. K. Wentz (Ed.), Continuing medical education:

Looking back, planning ahead (pp. 227–239). Hanover, NH: Dartmouth College Press.

Carrington, P. J., Scott, J., & Wasserman, S. (Eds.). (2005). Models and methods in social

network analysis. Cambridge, MA: Cambridge University Press.

Cazden, C. B. (1988). Classroom discourse. Portsmouth, NH: Heinemann.

246

Chang, R. W., Bordage, G., & Connell, K. J. (1998). Cognition, confidence, and clinical skills:

The importance of early problem representation during case presentations [Supplement].

Academic Medicine, 73(10), S109–111. Retrieved from

http://journals.lww.com/academicmedicine/

Charon, R. (2004). Narrative and medicine. The New England Journal of Medicine, 350(9), 862–

864. Retrieved from http://www.nejm.org

Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The

Journal of the Learning Sciences, 6(3), 271–315. doi:10.1207/s15327809jls0603_1

Chi, M. T. H. (2005). Commonsense conceptions of emergent processes: Why some

misconceptions are robust. Journal of the Learning Sciences, 14(2), 161–199.

doi:10.1207/s15327809jls1402_1

Christakis, N. A., & Fowler, J. H. (2007). The spread of obesity in a large social network over 32

years. New England Journal of Medicine, 357, 370–379.

Christakis, N. A., & Fowler, J. H. (2008). The collective dynamics of smoking in a large social

network. New England Journal of Medicine, 358, 2249–2258.

Christensen, C. R., Garvin, D. A., & Sweet, A. (Eds.). (1991). Education for judgment: The

artistry of discussion leadership. Boston, MA: Harvard Business School Press.

Chuy, M., Scardamalia, M., Bereiter, C., Prinsen, F., Resendes, M., Messina, R., . . . Chow, A.

(2010). Understanding the nature of scientific progress: A theory-building approach.

Canadian Journal of Learning and Technology, 36(1). Retrieved from http://www.cjlt.ca

Cimino, J. J. (1999). Development of expertise in medical practice. In R. J. Sternberg & J. A.

Horvath (Eds.), Tacit knowledge in professional practice: Researcher and practitioner

perspectives (pp. 102–120). Mahwah, NJ: Lawrence Erlbaum Associates.

College of Family Physicians of Canada. (n.d.). [Website]. Retrieved from http://cfpc.ca/CPD

Collins, J. (2009). Education techniques for lifelong learning: Lifelong learning in the 21st

century and beyond. RadioGraphics, 29, 613–622. doi:10.1148/rg.292085179

Collins, A., Brown, J. S., & Holum, A. (1991, Winter). Cognitive apprenticeship: Making

thinking visible. American Educator/American Federation of Teachers.

Collins, A., Brown, J. S., & Newman, S. (1989). Cognitive apprenticeship: Teaching the craft of

reading, writing, and mathematics. In L. B. Resnick (Ed.), Knowing, learning and

instruction: Essays in honor of Robert Glaser (pp. 453–494). Hillsdale, NJ: Lawrence

Erlbaum Associates.

Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and

methodological issues. The Journal of the Learning Sciences, 13(1), 15–42.

doi:10.1207/s15327809jls1301_2

247

Connell, K. J., Bordage, G., Gecht, M. R., & Chang, R. (1998). Assessing clinicians’ quality of

thinking and semantic competence: A training manual. Chicago, IL: University of

Illinois.

Cook, D. A. The research we still are not doing: An agenda for the study of computer-based

learning. (2005). Academic Medicine, 80(6), 541–548. Retrieved from http://journals

.lww.com/academicmedicine

Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2008).

Internet-based learning in the health professions. Journal of the American Medical

Association, 300(10), 1181–1196.

Cornelius, L. L., & Herrenkohl, L. R. (2004). Power in the classroom: How the classroom

environment shapes students’ relationships with each other and with concepts. Cognition

and Instruction, 22(4), 467–498. Retrieved from http://www.jstor

.org/stable/3233886

Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods

approaches. Thousand Oaks, CA: Sage.

Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five


Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods


CSCL2011. (2011, July 4–8). Ninth International Conference on Computer-Supported

Collaborative Learning, Hong Kong [Website]. Retrieved from http://www.isls.org

/cscl2011/index.htm?CFID=36331337&CFTOKEN=28690434

Curran, V., Lockyer, J., Sargeant, J., & Fleet, L. (2006). Evaluation of learning outcomes in web-

based continuing medical education. [Supplement]. Academic Medicine, 81(10), S30–

S34. doi:10.1097/01.ACM.0000236509.32699.f5

Custers, E. J. F. M., Regehr, G., & Norman, G. R. (1996). Mental representations of medical

diagnostic knowledge: A review [Supplement]. Academic Medicine, 71(10), S55–S61.

Retrieved from http://journals.lww.com/academicmedicine/

Dauphinee, W. D. (2011). The evolution of continuing medical education in Canada. In D. K.

Wentz (Ed.), Continuing medical education: Looking back, planning ahead (pp. 205–

217). Hanover, NH: Dartmouth College Press.

Dauphinee, W. D., & Wood-Dauphinee, S. (2004). The need for evidence in medical education:

The development of best evidence medical education as an opportunity to inform, guide,

and sustain medical education research. Academic Medicine, 79(10), 925–930. Retrieved

from http://journals.lww.com/academicmedicine/

Davis, D. (2011). Can CME save lives? The results of a Swedish, evidence-based continuing

education intervention. Annals of Family Medicine, 9, 198–200. doi: 10.1370/afm.1256

248

Davis, D., Barnes, B. E., & Fox, R. (Eds.). (2003). The continuing professional development of

physicians: From research to practice. Chicago, IL: American Medical Association

Press.

Davis, D., & Davis, N. (2010, February 9). Selecting educational interventions for knowledge

translation. Canadian Medical Association Journal, 182(2), E89–E93.

doi: 10.1503/cmaj.081241

Davis, D., Evans, M., Jadad, A., Perrier, L., Rath, D., Ryan, D., . . . Zwarenstein, M. (2003). The

case for knowledge translation: Shortening the journey from evidence to effect. British

Medical Journal, 327, 33–35. doi:10.1136/bmj.327.7405.33

Davis, D., Mazmanian, P. E., Fordis, M., Van Harrison, R., Thorpe, K. E., & Perrier, L. (2006).

Accuracy of physician self-assessment compared with observed measures of competence:

A systematic review. The Journal of the American Medical Association, 296(9), 1094–

1102. doi:10.1001/jama.296.9.1094

Davis, D., Thompson O’Brien, M. A., Freemantle, N., Wolf, F. M., Mazmanian, P., & Taylor-

Vaisey, A. (1999, September 1). Impact of formal continuing medical education: Do

conferences, workshops, rounds, and other traditional continuing education activities

change physician behavior or health care outcomes? Journal of the American Medical

Association, 282(9), 867–874.

Davis, N., Davis, D., & Bloch, R. (2008). Continuing medical education: AMEE Education

Guide #35. Medical Teacher, 30(7), 652–666. doi:10.1080/01421590802108323

Desa, V., Danjoux, C., Matyas, Y., Fitch, M., Husain, A., Horvath, N., . . . Barnes, E. A. (2008).

An interdisciplinary and collaborative initiative in palliative care research. Journal of

Pain Research, 2, 1–3. doi:10.2147/JPR.S4622

Dewey, J. (1997). Experience and education. New York, NY: Touchstone.

Dillman, D. (1978). Mail and telephone surveys: The total design method. New York, NY:

Wiley.

Dillman, D. (2000). Mail and internet surveys: The total design method. New York, NY: Wiley.

Donovan, S. M., Bransford, J. D., & Pellegrino, J. W. (Eds.). (1999). How people learn:

Bridging research and practice. Washington, DC: National Academy Press.

Dorman, T., & Miller, B. M. (2011). Continuing medical education: The link between physician

learning and health care outcomes. Academic Medicine, 86,1339.

doi: 10.1097/ACM0b013e3182308d49

Dreyfus, H. L., & Dreyfus, S. E. (1986). Five steps from expert to novice. In H. L. Dreyfus & S.

E. Dreyfus, with T. Athanasiou (Eds.), Mind over machine: The power of human intuition

and expertise in the era of the computer. (pp. 16–51). New York, NY: The Free Press.

Dunikowski, L. G. (2011). The College of Family Physicians of Canada: Continuing medical

education and continuing professional development in Canada. In D. K. Wentz (Ed.),

249

Continuing medical education: Looking back, planning ahead (pp. 240–247). Hanover,

NH: Dartmouth College Press.

Elstein, A. S. (1999). Heuristics and biases: Selected errors in clinical reasoning. Academic


Elstein, A. S., Shulman, L., & Sprafka, S. (1990). Medical problem solving: A ten-year

retrospective. Evaluation and the Health Professions, 13, 5–36.

End-of-Life Care Distance Education. (2004) [Website]. Retrieved from http://icarus.med

.utoronto.ca/eolcare/

Engeström, Y., Miettinen, R., & Punamäki, R.-L. (Eds.). (1999). Perspectives on activity theory.

Cambridge, MA: Cambridge University Press.

Eva, K. W., & Regehr, G. (2005). Self-assessment in the health professions: A reformulation and

research agenda [Supplement]. Academic Medicine, 80(10), S46–S54. Retrieved from


Fineberg, I. C., Wenger, N. S., & Forrow, L. (2004). Interdisciplinary education: Evaluation of a

palliative care training intervention for pre-professionals [Research report]. Academic


Fischer, G. (2009a). Democratizing design: New challenges and opportunities for computer-

supported collaborative learning. Proceedings (1) of CSCL’2009: Eighth International

Conference on Computer-Supported Collaborative Learning (pp. 282–286). University of

the Aegean, Rhodes, Greece. Retrieved from http://l3d.cs.colorado.edu

/~gerhard/papers/2009-CSCL-paper.pdf

Fischer, G. (2009b). End-user development and meta-design: Foundations for cultures of

participation. Proceedings of the Second International Symposium on End User

Development (pp. 3–14). Siegen, Germany. Retrieved from http://l3d.cs.colorado

.edu/~gerhard/papers/EUD-siegen-2009.pdf

Fischer, G. (2010, October). Extending boundaries with meta-design and cultures of

participation. Proceedings of the Sixth Nordic Conference on Human-Computer

Interaction, “Extending Boundaries” (pp. 168–177). Reykjavik, Iceland. Retrieved from

http://l3d.cs.colorado.edu/%7Egerhard/papers/2010/nordichi-paper.pdf

Fischer, G., & Giaccardi, E. (2006). Meta-design: A framework for the future of end user

development. In H. Lieberman, F. Paternò, & V. Wulf (Eds.), End user development:

Empowering people to flexibly employ advanced information and communication

technology (pp. 427–457). Dordrecht, the Netherlands: Kluwer Academic. Retrieved

from http://l3d.cs.colorado.edu/~gerhard/papers/EUD-meta-design-online.pdf

Fischer, G. & Konomi, S. (2007). Innovative socio-technical environments in support of

distributed intelligence and lifelong learning. Journal of Computer Assisted Learning,

23(4), 338–350. doi:10.1111/j.1365-2729.2007.00238.x

250

Fordis, M., King, J. E., Ballantyne, C. M., Jones, P. H., Schneider, K. H., Spann, S. J., &

Greisinger, A. J. (2006). Internet-based continuing medical education: Reply. Journal of

the American Medical Association, 295(7), 758–759. doi:10.1001/jama.295.7

.758-b

Foucault, M. (1972). The archaeology of knowledge and the discourse on language. New York,

NY: Pantheon Books

Foucault, M. (2001). Power (J. D. Faubion, Ed.; R. Hurley and others, Trans.). In P. Rabinow

(Series Ed.), The essential works of Foucault, 1954–1984: Vol. 3. New York, NY: New

Press.

Fowler, J. H., & Christakis, N. A. (2008). Dynamic spread of happiness in a large social

network: Longitudinal analysis over 20 years in the Framingham Heart Study. British

Medical Journal, 337. doi: 10.1136/bmj.a2338

Frank, J. R. (Ed). (2005). The CanMEDS 2005 physician competency framework. Better

standards. Better physicians. Better care. Ottawa, ON: The Royal College of Physicians

and Surgeons of Canada.

Freeman, L. C. (1978/1979). Centrality in social networks conceptual clarification. Social

Networks, 1(3), 215–239. doi:10.1016/0378-8733(78)90021-7

Friedman, M. H., Connell, K. J., Olthoff, A. J., Sinacore , J. M., & Bordage, G. (1998). Thinking

about student thinking: Medical student errors in making a diagnosis [Supplement].

Academic Medicine, 73(10), S19–S21. Retrieved from


Fujita, N., & Teplovs, C. (2009, June). Automating analysis of collaborative discourse:

Identifying idea clusters. Presentation at the International Conference of Computer

Supported Collaborative Learning, Rhodes, Greece.

Gardy, J. L., Johnston, J. C., Ho Sui, S. J., Cook, V. J., Sha, L., Brodkin, E., Rempel, S., . . .

Tang, P. (2011, June). Whole-genome sequencing and social-network analysis of a

tuberculosis outbreak. New England Journal of Medicine, 364,730–739. Retrieved from

http://www.nejm.org/doi/full/10.1056/NEJMoa1003176

Gettier, E. L. (1963). Is justified true belief knowledge? Analysis, 23, pp. 121–123. Retrieved

from http://www.ditext.com/gettier/gettier.html

Glick, M., & Holyoak, K. (1983). Schema induction and analogical transfer. Cognitive

Psychology, 15, 1–38.

Graham, I. D., Logan, J., Harrison, M. B., Straus, S. E., Tetroe, J., Caswell, W., & Robinson, N.

(2006). Lost in knowledge translation: Time for a map? The Journal of Continuing

Education in the Health Professions, 26(1),13–24. doi: 10.1002/chp.47

Granovetter, M. (1983). The strength of weak ties: A network theory revisited. Sociological

Theory, 1, 201–233.

251

Guest, C. B., Regehr, G., & Tiberius, R. G. (2001). The lifelong challenge of expertise. Medical

Education, 35(1), 78–81. doi:10.1111/j.1365-2923.2001.00831.x

Hager, M., Russell, S., & Fletcher, S.W. (Ed.). (2007, November 28–December 1). Continuing

education in the health professions—Improving healthcare through lifelong learning.

Proceedings of a conference sponsored by the Josiah Macy, Jr. Foundation, Bermuda.

New York, NY: Josiah Macy, Jr. Foundation. Retrieved from

http://josiahmacyfoundation.org/docs/macy_pubs/pub_ContEd_inHealthProf.pdf

Hanna, E., Premi, J., & Turnbull, J. (2000). Results of remedial continuing medical education in

dyscompetent physicians. Academic Medicine, 75(2), 174–176. Retrieved from


Haythornthwaite, C. (2002). Strong, weak, and latent ties and the impact of new media. The

Information Society, 18, 385–401.

Hewitt, J. (2001). Beyond threaded discourse. International Journal of Educational

Telecommunications, 7(3), 207–221.

Hewitt, J. (2005). Toward an understanding of how threads die in asynchronous computer

conferences. Journal of the Learning Sciences, 14(4), 567–589.

Hewitt, J., & Brett, C. (2007) The relationship between class size and online activity patterns in

asynchronous computer conferencing environments. Computers & Education, 49, 1258–

1271.

Hewitt, J., & Scardamalia, M. (1998). Design principles for distributed knowledge building

processes. Educational Psychology Review, 10(1), 75–96. doi:

10.1023/A:1022810231840

Hewitt, J., & Teplovs, C. (1999). An analysis of growth patterns in computer conferencing

threads [Article 29]. Proceedings of the 1999 conference on Computer Support for

Collaborative Learning. Retrieved from http://portal.acm.org/citation

.cfm?id=1150269&CFID=35298853&CFTOKEN=79054397

Hmelo-Silver, C. E., & Barrows, H. S. (2008). Facilitating collaborative knowledge building.

Cognition and Instruction, 26,(1), 48–94.

Hodges, B. D., Kuper, A., & Reeves, S. (2008). Qualitative research: Discourse analysis. British

Medical Journal, 337, 570–572. doi:10.1134/bmja879

Holyoak, K., & Thagard, P. (1995). Mental leaps: Analogy in creative thought. Cambridge, MA:

MIT Press/Bradford Books.

Hong, H-Y., Scardamalia, M., & Zhang, J. (2010). Knowledge society network: Toward a

dynamic, sustained network for building knowledge. Canadian Journal of Learning and

Technology, 36(1). Retrieved from http://www.cjlt.ca

Jones, G. A., McCarney, P. L., & Skolnik, M. L. (Eds.). (2005). Creating knowledge,

strengthening nations: The changing role of higher education. Toronto, ON: University

of Toronto Press.

252

Kanter, S. (2008, August). Toward better descriptions of innovations. Academic Medicine, 83(8),

703–704.

Keating, N. L., Ayanian, J. Z., Cleary, P. D., & Marsden, P. V. (2007, June). Factors affecting

influential discussions among physicians: A social network analysis of a primary care

practice. Journal of General Internal Medicine, 22(6), 794–796.

Knowles, M. S. (1975). Self-directed learning: A guide for learners and teachers. Englewood

Cliffs, NJ: Prentice Hall Regents.

Kolodner, J. (1992). An introduction to case-based reasoning. Artificial Intelligence Review,

6(1), 3–34. doi:10.1007/BF00155578

Koschmann, T. D., Myers, A. C., Feltovich, P. J., & Barrows, H. S. (1994). Using technology to

assist in realizing effective learning and instruction: A principled approach to the use of

computers in collaborative learning. The Journal of the Learning Sciences, 3(3), 227–

264. doi:10.1207/s15327809jls0303_2

Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing

one’s own incompetence lead to inflated self-assessments. Journal of Personality and

Social Psychology, 77(6), 1121–1134.

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation.

Cambridge, UK: Cambridge University Press.

Lax, L., Scardamalia, M., Watt-Watson, J., Hunter, J., & Bereiter, C. (2010). Beyond learning

management systems: Designing for interprofessional knowledge building in the health

sciences. Canadian Journal of Learning and Technology, 36(1). Retrieved from

http://www.cjlt.ca

Lax, L., Singh, A., Scardamalia, M., & Librach, L. (2006). Self-assessment for knowledge

building in health care. QWERTY: Interdisciplinary Journal of Technology, Culture and

Education. 1(2), 19–37. Retrieved from http://www.ckbg.org

Lax, L., Watt-Watson, J., Pennefather, P., Hunter, J., & Scardamalia, M. (2002). Project to

examine the cost/effectiveness of e-learning for interfaculty health sciences education.

Unpublished report, funded by The Academic Priority Fund of the Council of Health

Science and Social Work Deans and the University of Toronto Centre for the Study of

Pain, Toronto, ON. Retrieved from http://www.ikit.org/fulltext/Lax_PainWeek

_FinalReport_2002.pdf

Lax, L., Watt-Watson, J., Pennefather, P., Hunter, J. & Scardamalia, M. (2003a, April). E-

learning for strategic clinical management: Advancing beyond identification of students’

conceptions / misconceptions. Poster session presented at the annual meeting of the

Association of Canadian Medical Colleges/Canadian Association for Medical Education,

Quebec City, QC.

Lax, L., Watt-Watson, J., Pennefather, P., Hunter, P., & Scardamalia, M. (2003b). The pain week

e-learning project: An undergraduate interprofessional knowledge building initiative

[Supplement 1]. The Journal of Pain, 4(2), 726.

253

Lee, E. Y. C., Chan, C. K. K., & Van Aalst, J. (2006). Students assessing their own collaborative

knowledge building. International Journal of Computer-Supported Collaborative

Learning, 1, 277–307. Retrieved from http://www.springerlink.com/

Lemieux, M., & Bordage, G. (1992). Propositional versus structural semantic analyses of

medical diagnostic thinking. Cognitive Science, 16, 185–204.

doi:10.1207/s15516709cog1602_2

Lester, H., & Tritter, J. Q. (2001). Medical error: A discussion of the medical construction of

error and suggestions for reforms of medical education to decrease error. Medical

Education, 35(9), 855–861. doi:10.1046/j.1365-2923.2001.01003.x

Liao, S., Alpesh, A., & Rucker, L. (2004). An innovative, longitudinal program to teach residents

about end-of-life care. Academic Medicine, 79(8), 752–757. Retrieved from


Ligorio, M. B. (2009). Identity as a product of knowledge building: The role of mediated

dialogue. QWERTY: Interdisciplinary Journal of Technology, Culture and Eduation,

4(1), 33–46. Retrieved from www.ckbg.org/qwerty/indiex.php/qwerty/article/view/46

Ligorio, M. B. (2010). Dialogical relationship between identity and learning. Culture &

Psychology, 16(1), 93–107. doi: 10.1177/1354067X09353206

Lipponen, L., Rahikainen, M., Lallimo, J., & Hakkarainen, K. (2003). Patterns of participation

and discourse in elementary students’ computer-supported collaborative learning.

Learning and Instruction, 13, 487–509.

Mamede, S., & Schmidt, H. G. (2004). The structure of reflective practice in medicine. Medical

Education, 38, 1302–1308. doi:10.1111/j.1365-2929.2004.01917.x

Mandil, S. (1982). The meaning and role of communication networks in health care with regard

to WHO’s goal of “Health for all by the year 2000.” In H. E. Peterson & A. I. Isakkson

(Eds.), Communication networks in health care (pp. 7–42). Amsterdam, the Netherlands:

North-Holland.

Manning, P. R., & DeBakey, L. (2011). Lifelong medical education: Past, present, future. In D.

K. Wentz (Ed.), Continuing medical eduation: Looking back, planning ahead (pp. 17–

20). Hanover, NH: Dartmouth College Press.

Marin, A., & Wellman, B. (2011). Social network analysis: An introduction. In J. Scott & P.

Carrington (Eds.), The Sage handbook of social network analysis (pp. 11–25). London,

UK: Sage.

Marinopoulos, S. S., Dorman, T., Ratanawongsa, N., Wilson, L. M., Ashar, B. H., Magaziner, J.

L., Miller, R. G., . . . Bass, E. B. (2007, January). Effectiveness of continuing medical

education (Evidence Report/Technology Assessment No. 149, Prepared by the Johns

Hopkins Evidence-based Practice Centre, under Contract No. 2900-02-0018, AHRQ

Publication No. 07-E006). Rockville, MD: Agency for Healthcare Research and Quality.

254

Mayer, R. E. (2005). Cognitive theory of multimedia learning. In Mayer, R. E. (Ed.), The

Cambridge handbook of multimedia learning (pp. 31–48). Cambridge: University Press.

Mazmanian, P. E., & Davis, D. A. (2002). Continuing medical education and the physician as

learner: Guide to the evidence. Journal of American Medical Association, 288(9), 1057–

1060. doi:10.1001/jama.288.9.1057

Merriam, S. B. (2001). Qualitative research and case study applications in education. San

Francisco, CA: Jossey-Bass.

Miller, L. (2011). Using social networks to create ACOs (Accountable Care Organizations).

Retreived from http://www.hhnmag.com/hhnmag/HHNDaily/HHNDailyDisplay

.dhtml?id=3230009892

Miller, S. H., Thompson, J. N., Mazmanian, P. E., Aparicio, A., Davis, D. A., Spivey, B. E., &

Kahn, N. B. (2008). Continuing medical education, professional development, and

requirements for medical licensure: A white paper of the Conjoint Committee on

Continuing Medical Education. The Journal of Continuing Education in the Health

Professions, 28(2), 95–98. Retrieved from http://www.jcehp. com/

Mills, S. (1997). Discourse in the new critical idiom. (J. Drakakis, Series Ed.). New York, NY:

Routledge.

Moores, L. K., Dellert, E., Baumann, M. H., & Rosen, M. J. (2009). Effectiveness of continuing

medical education: American College of Chest Physicians Evidence-Based Educational

Guidelines. Chest, 135, 1S–4S. doi:10.1378/chest.08-2511

Morrison, R. S., & Meier, D. E. (2004). Palliative care. The New England Journal of Medicine,

350, 2582–2590. doi:10.1056/NEJMcp035232#t

Moss, J., & Beatty, R. (2010). Knowledge building and mathematics: Shifting the responsibility

for knowledge advancement and engagement. The Canadian Journal of Learning and

Technology, 36(1). Retrieved from http://www.cjlt.ca

Murad, M. H., Coto-Yglesia, F., Varkey, P., Prokop, L., & Murad, A. L. (2010). The

effectiveness of self-directed learning in health professions education: A systematic

review. Medical Education, 44, 1057–1068. doi: 10.1111/j.1365-2923.2010.03750.x

Mylopoulos, M. (2007). Implicit theories of innovating and expertise (Doctoral dissertation).

Retrieved from http://openlibrary.org/works/OL12753654W/Implicit_theories_of

_innovation_and_expertise

Mylopoulos, M., & Scardamalia, M. (2008). Doctors’ perspectives on their innovations in daily

practice: Implications for knowledge creation in health care. Medical Education, 42, 975–

981.

Nendaz, M. R., Junod, A. F., Vu, N. V., & Bordage, G. (1998). Medical students and housestaff:

Partners in learning: Eliciting and displaying diagnostic reasoning during educational

rounds in internal medicine: Who learns from whom? [Supplement]. Academic Medicine,

73(10), S54–S56. Retrieved from http://journals.lww.com/academicmedicine

255

Netminer 3. (n.d.) [Computer software]. Available from http:www.netminer.com

Newell, A. (1980). Reasoning, problem solving and decision processes: The problem space as a

fundamental category. In R. Nickerson (Ed.), Attention and performance VIII (pp. 693–

718). Hillsdale, NJ: Lawrence Erlbaum Associates.

Ng, S. K., & Law, N. W. Y. (2003). Knowledge building in the absence of teacher facilitation. In

Y. S. Chee, N.W. Y. Law, K. T. Lee, & D. Suthers (Eds.), Proceedings of the

International Conference on Computers in Education 2003 (pp. 406–413). Norfolk, VA:

Association for the Advancement of Computing in Education.

Nonaka, I. (1991, November/December). The knowledge-creating company. The Harvard

Business Review, 96–104.

Norman, G. R., & Schmidt, H. G. (1992). The psychological basis of problem-based learning: A

review of the evidence. Academic Medicine, 67(9), 557–565. Retrieved from


Organization for Economic Co-operation and Development (OECD). (2007, April). Participative

Web: User-created content. Retrieved from http://www.oecd.org

/dataoecd/57/14/38393115.pdf

Palincsar, A., & Brown, A. (1984). Reciprocal teaching of comprehension-fostering and

comprehension-monitoring activities. Cognition and Instruction, 1, 117–175.

Patel, V. L., Arocha, J. F., & Kaufman, D. R. (1999). Expertise and tacit knowledge in medicine.

In R. J. Sternberg & J. A. Horvath (Eds.), Tacit knowledge in professional practice:

Researcher and practitioner perspectives (pp. 75–120). Mahwah, NJ: Lawrence Erlbaum

Associates.

Patel, V. L., Yoskowitz, N. A., Arocha, J. F., & Shortliffe, E. H. (2009). Cognitive and learning

sciences in biomedical and health instructional design: A review with lessons for

biomedical informatics education. Journal of Biomedical Informatics, 42(1), 176–197.

doi:10.1016/j.jbi.2008.12.002

Pearls™. (n.d.) [Computer software]. Available from http://www.cfpc.ca/Pearls

Pereira, J., Palacios, M., Collin, T., Wedel, R., Galloway, L., Murray, . . . Lockyer, J. (2008).

The impact of a hybrid online and classroom-based course on palliative care

competencies of family medicine residents. Palliative Medicine, 22(8), 1–9.

doi:10.1177/0269216308094561

Petroski, H. (1996). Invention by design. Cambridge, MA: Harvard University Press.

Philip, D. N. (2010). Social network analysis to examine interaction patterns in knowledge

building communities. Canadian Journal of Learning and Technology, 36(1). Retrieved

from http://www.jclt.ca

Pisacane, A. (2008). Rethinking continuing medical education. British Medical Journal, 337,

a973. doi:10.1136/bmj.a973

256

Polanyi, M. (1983). The tacit dimension. Gloucester, MA: Peter Smith.

Popper, K. R. (1972). Objective knowledge: An evolutionary approach. London, UK: Oxford

University Press.

Porter-Williamson, K., von Gunten, C. F., Garman, K., Herbst, L., Bluestein, H. G., & Evans, W.

(2004). Improving knowledge in palliative medicine with a required hospice rotation for

third-year medical students. Academic Medicine, 79(8), 777–782. Retrieved from


Posavac, E. J., & Carey, R. G. (2003). Program evaluation: Methods and case studies. Upper

Saddle River, NJ: Prentice Hall.

Price, D., Havens, C., & Bell, M. J. (2011). Continuing professional development and

improvement to meet current and future continuing medical education needs of

physicians. In Wentz, D. K. (Ed.), Continuing medical eduation: Looking back, planning

ahead (pp. 1–14). Hanover, NH: Dartmouth College Press.

Punja, Z. (2007). The role of Knowledge Building in medical education (Unpublished doctoral

dissertation). OISE/UT, Toronto, ON.

Quality End-of-Life Care Coalition of Canada. (2005, September). Framework for a national

strategy on palliative and end-of-life care. Retrieved from http://www.chac.ca/advocacy

/briefs/docs/palliative_brief_qelccc.pdf

Regehr, G., & Eva, K. (2006). Self-assessment, self-direction, and the self-regulating

professional. Clinical Orthopaedics and Related Research, 449, 34–38.

doi:10.1097/01.blo.0000224027.85732.b2

Regehr, G., & Norman, G. (1996). Issues in cognitive psychology: Implications for professional

education. Academic Medicine, 71(9), 988–1001. Retrieved from http://journals.lww.com

/academicmedicine

Romanow, R. C. (2002). Building on values: Commission on the future of health care in Canada

(Final report). Ottawa, ON: Commission on the Future of Health Care in Canada.

Retrieved from www.hc-sc.gc.ca/english/care/romanow/indexl.html

Russell, A. (2005). Progress towards design of a knowledge building community in health care

(Unpublished doctoral dissertation). OISE/UT, Toronto, ON.

Russell, A., Campbell, H., Scardamalia, M., & Bereiter, C. (2002). Fostering interprofessional

practice in rehabilitation and complex continuing care through collaboration and virtual

learning. Retrieved from http:// ikit.org/OLT.html

Saljo, R. (2003). Representational tools and the transformation of learning. In B. Wasson, S.

Ludvigsen, & U. Hoppe (Eds.), Designing for change (pp. 1–2). Retrieved from

www.springer.com/?SGWID=4-102-45-131802-0

Salomon, G. (1994a). Differences in patterns: Studying computer enhanced learning

environments. In E. Vosniadou, E. De Corte, & H. Mandl (Eds.), Technology-based

257

learning environments: Psychological and educational foundations (pp. 79–88). Berlin,

Germany: Springer-Verlag

Salomon, G. (1994b). Interaction of media, cognition, and learning. Hillsdale, NJ: Lawrence

Erlbaum Associates.

Sargeant, J., Mann, K., van der Vleuten, C., & Metsemakers, J. (2008). Directed self-assessment:

Practice and feedback within a social context. The Journal of Continuing Education in

the Health Professions, 28(1), 47–54. Retrieved from http://www.jcehp. com

Sawyer, R. K. (Ed.), (2006). The Cambridge handbook of the learning sciences. Cambridge, UK:

Cambridge University Press.

Scardamalia, M. (1999). Moving ideas to the centre. In L. Harasim (Ed.), Wisdom and wizardry:

Celebrating the pioneers of online education (pp. 14–15). Vancouver, BC: Telelearning.

Scardamalia, M. (2002). Collective cognitive responsibility for the advancement of knowledge.

In B. Smith (Ed.), Liberal education in a knowledge society (pp. 67–98). Chicago, IL:

Open Court.

Scardamalia, M. (2003a). Knowledge building. (Advances beyond CSILE). [Supplement 3,

“Learning Technology Innovation in Canada”]. Journal of Distance Education, 17, 10–

14.

Scardamalia, M. (2003b). Knowledge society network (KSN): Toward an expert society for

democratizing knowledge [Supplement 3, “Learning Technology Innovation in Canada”].

Journal of Distance Education, 17, 63–66.

Scardamalia, M. (2004a). CSILE/Knowledge forum. In A. Kovalchick, & K. Dawson (Eds.),

Educational technology: An encyclopedia (pp. 183–192). Santa Barbara, CA: ABC-CILO

Scardamalia, M. (2004b). Instruction, learning, and knowledge building: Harnessing theory,

design, and innovation dynamics. Educational Technology, 44(3), 30–33.

Scardamalia, M., & Bereiter, C. (1991). Higher levels of agency for children in knowledge

building: A challenge for the design of new knowledge media. The Journal of the

Learning Sciences, 1(1), 37–68. doi:10.1207/s15327809jls0101_3

Scardamalia, M., & Bereiter, C. (1994). Computer support for knowledge-building communities.

The Journal of the Learning Sciences, 3(3), 265–283. doi:10.1207/s15327809jls0303_3

Scardamalia, M., & Bereiter, C. (1996). Student communities for the advancement of

knowledge. Communications of the ACM, 39(4), 36–37.

Scardamalia, M., & Bereiter, C. (2003a). Knowledge building. In J. W. Guthrie (Ed.),

Encyclopedia of Education (2nd ed., pp. 1370–1373). New York, NY: Macmillan

Reference Library.

Scardamalia, M., & Bereiter, C. (2003b). Knowledge building environments: Extending the

limits of the possible in education and knowledge work. In A. DiStefano, K. E.

258

Rudestam, & R. Silverman (Eds.), Encyclopedia of distributed learning (pp. 269–272).

Thousand Oaks, CA: Sage.

Scardamalia, M. & Bereiter, C. (2005). Does education for the knowledge age need a new

science? European Journal of School Psychology, 3(1), 21–40.

Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and

technology. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences

(pp. 97–115). New York, NY: Cambridge University Press.

Scardamalia, M., & Bereiter, C. (2010). A brief history of knowledge building. Canadian

Journal of Learning and Technology, 36(1). Retrieved from http://www.cjlt.ca

Scardamalia, M., Bereiter, C., McLean, R., Swallow, J., & Woodruff, E. (1989). Computer-

supported intentional learning environments. Journal for Educational Computing

Research, 5(1), 51–68. doi:10.2190/CYXD-6XG4-UFN5-YFBO

Scardamalia, M., Bransford, J., Kozma, R., & Quellmalz, E. (in press). New assessments and

environments for knowledge building. In P. Griffin, B. McGaw, & E. Care (Eds.),

Assessment and learning of 21st Century Skills (pp. 231–301). New York, NY: Springer.

Retrieved from http://atc21s.org/default.aspx

Schmidt, H. G., & Boshuizen, H. P. A. (1992). Encapsulation of biomedical knowledge. In D. A.

Evans & V. L. Patel (Eds.), Advanced models of cognition for medical training and

practice (pp. 265–282). Berlin, Germany: Springer-Verlag.

Schmidt, H. G., Norman, G. R., & Boshuizen, H. P. A. (1990). A cognitive perspective on

medical expertise: Theory and implications. Academic Medicine, 65(10), 611–621.

Erratum published April1992, in Academic Medicine, 67(4), 287. Retrieved from


Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York,

NY: Basic Books.

Schön, D. A. (1987). Educating the reflective practitioner: Toward a new design for teaching

and learning in the professions. San Francisco, CA: Jossey-Bass.

Schuwirth, L., & van der Vleuten, C. (2004). Merging views on assessment. Medical Education,

38, 1208–1210. doi:10.1111/j.1365-2929.2004.02055.x

Scott, J. (2001). Power. Cambridge, UK: Polity Press.

Scott, J. (2009). Social network analysis: A handbook (2nd ed.). Thousand Oaks, CA: Sage.

Scott, J., & Carrington, P. (Eds.). (2011). The Sage handbook of social network analysis.

London, UK: Sage.

Self Learning™. (n.d.) [Computer software]. Available from http://www.cfpc.ca/sli

259

Sha, L., & van Alst, J. (2003, April 21–25). An application of social network analysis to

knowledge building. Proceedings of the Annual Meeting of the American Educational

Research Association. Chicago, IL.

Sha, L., Teplovs, C., & van Aalst, J. (2010, June). A visualization of group cognition: Semantic

network analysis of a CSCL community. In K. Gomez, L. Lyons, & J. Radinsky (Eds.),

Learning in the disciplines: Proceedings of the Ninth International Conference of the

Learning Sciences (Vol. 1, pp. 929–936). Chicago, IL: International Society of the

Learning Sciences.

Sibbald, D. J. (2009). Potential for knowledge building in large size pharmacy classrooms

(Doctoral dissertation). Retrieved from Library and Archives Canada. (ISBN: 978-0-494-

61091-6)

Silver, I., Campbell, C., Marlow, B., & Sargeant, J. (2008). Self-assessment and continuing

professional development: The Canadian perspective. The Journal of Continuing

Education in the Health Professions, 28(1), 25–31. Retrieved from http://www.jcehp

.com/

Silverdale, N., & Katz, J. (2005). The impact of a distance learning death and dying course: An

analysis of student self-reported changes. Nurse Education Today, 25(7), 509–518.

doi:10.1016/j.nedt.2005.05.004

Singer, P. A., & Bowman, K. W. (2002). Quality end-of-life care: A global perspective.

Palliative Care, 1(4). doi: 10.1186/1472-684X-1-4

Singer, P. A., Martin, D. K., Kelner, M. (1999). Quality end of life care: Patients’ perspectives.

Journal of the American Medical Association, 281, 163–168.

Spiro, R. J., Coulson, R. L., Feltovich, P. J., & Anderson, D. (1988). Cognitive flexibility theory:

Advanced knowledge acquisition in ill-structured domains. In V. Patel (Ed.), Proceedings

of the 10th Annual Conference of the Cognitive Science Society (pp. 433–439). Hillsdale,

NJ: Lawrence Erlbaum Associates.

Spiro, R. J., Feltovich, P. J., Jacobson, M. J., & Coulson, R. L. (1991). Cognitive flexibility,

constructivism, and hypertext: Random access instruction for advanced knowledge

acquisition in ill-structured domains. Educational Technology, 31(5), 24–33.

Spiro, R. J., & Jehng, J. C. (1991). Cognitive flexibility and hypertext: Theory and technology

for the nonlinear and multidimensional traversal of complex subject matter. In D. Nix &

R. Spiro (Eds.), Cognition, education, and multimedia: Exploring ideas in high

technology (pp. 163–173). Mahwah, NJ: Lawrence Erlbaum Associates.

Stokes, D. (1997). Pasteur’s quadrant basic science and technological innovation. Washington,

DC: The Brookings Institution.

Sullivan, A. M., Warren, A. G., Lakoma, M. D., Liaw, K. R., Hwang, D., & Block, S. D. (2004).

End-of-life care in the curriculum: A national study of medical education deans

[Research report]. Academic Medicine, 79(8), 760–768. Retrieved from http://journals

.lww.com/academicmedicine

260

Sutherland, S. (2011, January 27). Education rounds. Retrieved from http://www.med.uottawa.ca

/aime/eng/education_rounds.html

Sutherland, S., Moineau, G., Leddy, J., Cohen, E., Hamstra, S., & Regehr, G. (2011, April).

Understanding the utility of social network analysis for the medical trainee. AIME 4th

Annual Medical Education Day (p. 5). University of Ottawa, Academy for Innovation in

Medical Education. Retrieved from http://www.med.uottawa.ca/aime/assets

/documents/aime_day_4_abstracts.pdf

Tabak, I., & Baumgartner, E. (2004). The teacher as partner: Exploring participant structures,

symmetry, and identity work in scaffolding. Cognition and Instruction, 22(4), 393–

429.doi: 10.1207/s1532690Xci2204_2

Teplovs, C. (2010). Visualization of knowledge spaces to enable concurrent, embedded, and

transformative input to knowledge building processes (Unpublished doctoral

dissertation). OISE/UT, Toronto, ON.

Teplovs, C., & Fujita, N. (2009). Determining curricular coverage of student contributions to an

online discourse environment: Using latent semantic analysis to construct differential

term clouds. In C. O’Malley, D. Suthers, P. Reimann, & A. Dimitracopoulou (Eds.),

Computer supported collaborative learning practices (pp.165–167). Rhodes, Greece:

International Society of the Learning Sciences.

Teplovs, C., & Scardamalia, M. (2007). Visualizations for knowledge building assessment. Paper

presented at the AgileViz workshop, Computer Support for Collaborative Learning

Conference. Retrieved from http://chris.ikit.org/agilevizcscl-teplovs-scardamalia.pdf

Thagard, P. (1997). Medical analogies: Why and how. In P. Langley & M. Shafto (Eds.),

Proceedings of the Nineteenth Annual Conference of the Cognitive Science Society (pp.

739–744). Mahwah, NJ: Lawrence Erlbaum Associates.

Thagard, P. (2000). Coherence in thought and action. Cambridge, MA: MIT Press.

Thagard, P. (2007, January). Coherence, truth, and the development of scientific knowledge.

Philosophy of Science, 74, 28–47. Retrieved from http://cogsci.uwaterloo.ca/Articles

/coherence.truth.pos.2007.pdf

doi:10.1086/520941

Thagard, P., & Shelley, C. (1997) Abductive reasoning: Logic, visual thinking, and coherence.

In: M.-L. Dalla Chiara et al (eds), Logic and scientific methods (pp. 413–427). Dordrecht,

the Netherlands: Kluwer Academic. Retrieved from http://cogprints.org/671/1

/FAbductive.html

University of Toronto, Faculty of Medicine, Office of Continuing Education & Professional

Development. (2011). Strategic academic plan 2011–2016. Retrieved from http://www

.facmed.utoronto.ca/Page14.aspx

Valente, T. W. (2010). Social networks and health: Models, methods and applications. Oxford,

UK: Oxford University Press.

261

Viilo, M., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2011). Supporting the technology-

enhanced collaborative inquiry and design project: A teacher’s reflections on practices.

Teachers and Teaching Theory and Practice, 17, 51–72.

Von Hippel, E. (2005). Democratizing innovation [E-book, licensed under a Creative Commons

License]. Retrieved from http://www.mit.edu/people/evhippel/democ1.htm

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes.

Cambridge, MA: Harvard University Press.

Walton, J. M., & Oswald, A. (2011, May 7–11). Look who’s talking: Patterns of interaction in

problem-based learning. Scholarship in Medical Education, Canadian Conference in

Medical Education, Toronto, ON. Retrieved from http://www.mededconference.ca

/documents/Program2011_eng_WEB.pdf

Wang, Y., & Li, X. (2007). Social network analysis of interaction in online learning

communities. Seventh IEEE International Conference on Advanced Learning

Teachnologies, Niigata, Japan.

Wang, L. (2010). How social network position relates to knowledge buildng in online learning

communities. Frontiers of Education in China, 5(1), 4–25. doi: 10.1007/s11516-010-

000304

Wasserman, S., & Faust, K. (1994). Social network analysis: methods and applications.

Cambridge, UK: Cambridge University Press.

Watt-Watson, J., Hunter, J., Pennefather, P., Librach, L., Raman-Wilms, L., Schreiber, M.,

. . . Salter, M. (2004). An integrated undergraduate pain curriculum based on IASP

curricula, for six health sciences facilities. Pain, 110, 140–148.

Wear, D. (2002). “Face-to-face with it”: Medical students’ narratives about their end-of-life

education. Academic Medicine, 77(4), 271–277. Retrieved from http://journals.lww

.com/academicmedicine

Weissman, D. E., & Block, S. D. (2002). ACGME requirements for end-of-life training in

selected residency and fellowship programs. Academic Medicine, 77(4), 299–304.

Retrieved from http//journals.lww.com/academicmedicine

Wellman, B. (1983). Network analysis: Some basic principles. Sociological Theory, 1, 155–200.

Retrieved from http://www.jstor.org/stable/202050

Wellman, B. (1988). Structural analysis: From method and metaphor to theory and substance. In

B. Wellman & S. D. Berkowitz (Eds.), Social structures: A network approach (pp. 19–

61). Cambridge, UK: Cambridge University Press.

Wenger, E. C. (1998). Communities of practice: Learning, meaning, and identity. Cambridge,

UK: Cambridge University Press.

Wentz, D. K. (Ed.) Continuing medical education: Looking back, planning ahead. Hanover, NH:

Dartmouth College Press.

262

Whitehead, A. N. (1933). Adventures of ideas. New York, NY: The Free Press.

Whitehead, A. N. (1967). The aims of education and other essays. New York, NY: The Free

Press.

World Health Organization (WHO). (1990). Cancer pain relief and palliative care (World

Health Organization Technical Report, Series 804). Geneva, Switzerland: Author.

Wood, E. W., Meekin, S. A., Fins, J. J., & Fleischman, A. R. (2004, April). Enhancing palliative

care education in medical school curricula: Implementation of the palliative education

assessment tool. Academic Medicine, 77(4), 285–291.

Woollard, R. (2011). A history of the committee on the accreditation of continuing medical

education—Canada. In D. K. Wentz (Ed.), Continuing medical education: Looking back,

planning ahead (pp. 218–226). Hanover, NH: Dartmouth College Press.

Zhang, J., Scardamalia, M., Lamon, M., Messina, R., & Reeve, R. (2007). Socio-cognitive

dynamics of knowledge building in the work of 9- and 10-year-olds. (2007, April).

Educational Technology, Research and Development, 55(2),117–145. doi:

10.1007/s11423-006-9019-0

Zhang, J., Scardamalia, M., Lamon, M., Messina, R., Reeve, R., & Teplovs, C. (2005, April 11–

15). How does knowledge work in classrooms: Knowledge dynamics in online knowledge

building environment. Paper presented at the American Educational Research Association

annual meeting, Montreal, QC.

Zhang, J., Scardamalia, M., Reeve, R., & Messina, R. (2009). Designs for collective cognitive

responsibility in knowledge-building communities. The Journal of the Learning Sciences,

18(1), 7–44. doi: 10.1080/10508400802581676

263

APPENDICES

264

APPENDIX A

Information Letter and Informed Consent Form

continued…

266

APPENDIX B

End-of-Life Care Distance Education Program Homepage and Schedule

267

APPENDIX C

End-of-Life Care Distance Education Program Modules in Knowledge Forum®

and Multimedia Case Note

268

APPENDIX D

Pain Pre-/Posttest Item Analysis 2004/2005

EOLC Pain Pre-/Posttest

2004-05 Item Analysis (N = 9)

Question Objective Sig Sig Mean Mean

2-tailed 1-tailed Pre Post Diff

q1 1 1.000 0.500 89 100 11

q2 1 na na 100 100 0

q3 3 na na 100 100 0

q4 3 0.625 0.313 56 78 22

q5 2 0.250 0.125 33 67 33

q6 2 0.500 0.250 67 89 22

q7 2 1.000 0.500 0 11 11

q8_1 2 1.000 0.500 22 33 11

q8_2 2 0.125 0.063 56 100 44

q8_4 2 1.000 0.500 33 44 11

q8_5 2 1.000 0.500 56 56 0

q8_7 2 1.000 0.500 89 100 11

q9_1 2 0.375 0.188 11 44 33

q9_2 2 0.063 0.031 33 89 56

q9_4 2 1.000 0.500 11 22 11

q9_5 2 1.000 0.500 44 56 11

q10_1 2 na na 0 0 0

q10_2 2 1.000 0.500 22 11 -11

q10_4 2 na na 0 0 0

q10_5 2 0.250 0.125 33 67 33

q11_1 2 0.625 0.313 33 11 -22

q11_2 2 0.016 0.008 78 0 -78

q11_4 2 1.000 0.500 11 0 -11

q11_5 2 1.000 0.500 56 56 0

q11_7 2 1.000 0.500 89 100 11

q12 2 1.000 0.500 22 33 11

q13 4 0.250 0.125 11 44 33

q14 4 0.125 0.063 56 100 44

q15 1 1.000 0.500 89 100 11

q16 1 0.500 0.250 67 89 22

q17 1 na na 100 100 0

269

q18 2 1.000 0.500 22 11 -11

q19 2 1.000 0.500 89 89 0

q20 2 1.000 0.500 78 78 0

q21 3 0.250 0.125 33 67 33

q22_1 3 1.000 0.500 67 78 11

q22_2 3 1.000 0.500 67 78 11

q22_3 3 1.000 0.500 89 89 0

q22_4 3 1.000 0.500 67 67 0

q22_5 3 1.000 0.500 67 78 11

q22_6 3 1.000 0.500 67 78 11

q22_7 3 1.000 0.500 56 67 11

q23 4 0.500 0.250 78 100 22

q24 4 0.250 0.125 67 100 33

q25 1 na na 100 100 0

q26 4 na na 0 0 0

q27 4 1.000 0.500 67 78 11

q28 4 0.500 0.250 78 100 22

q29 4 0.250 0.125 67 100 33

q30 4 1.000 0.500 78 89 11

q31_1 3 1.000 0.500 89 100 11

q31_2 3 1.000 0.500 89 100 11

q31_3 3 0.500 0.250 78 100 22

q31_4 3 0.625 0.313 67 89 22

q31_5 3 1.000 0.500 89 89 0

q31_6 3 na na 100 100 0

q32_1 4 na na 100 100 0

q32_2 4 na na 100 100 0

q32_3 4 1.000 0.500 89 100 11

q32_4 4 na na 100 100 0

q32_5 4 1.000 0.500 89 100 11

q32_6 4 na na 100 100 0

270

Note: EoL Care 2004/2005 Items with Positive Change from Pre- to Posttest

271

Note: EoL Care 2004/2005 Items with Zero or Negative Pre- to Posttest Change

272

APPENDIX E

Pain Pre-/Posttest Item Analysis 2005/2006

EOLC Pain Pre-/Posttest 2005-06

Item Analysis (N=11)

Question

Objectives

Pre-test

(%)

Post-test

(%)

Exact Sig.

(2-tailed) 1-tailed

Post-Pre

% Diff.

Q1 1 73 91 .500 .250 18

Q2 1 73 82 1.000 .500 9

Q3 3 100 100 na na 0

Q4 3 91 91 1.000 .500 0

Q5 2 18 64 .063 .031 45

Q6_1 2 100 100 na na 0

Q6_2 2 100 100 na na 0

Q6_3 2 100 100 na na 0

Q6_4 2 82 100 .500 .250 18

Q7 2 91 82 1.000 .500 -9

Q8 2 18 36 .500 .250 18

Q9_1 2 82 82 1.000 .500 0

Q9_2 2 91 100 1.000 .500 9

Q9_3 2 55 64 1.000 .500 9

Q10_1 2 73 73 1.000 .500 0

Q10_2 2 91 100 1.000 .500 9

Q10_3 2 82 73 1.000 .500 -9

Q11_1 2 18 36 .625 .313 18

Q11_2 2 73 73 1.000 .500 0

Q11_3 2 36 55 .500 .250 18

Q12_1 2 82 91 1.000 .500 9

Q12_2 2 91 91 1.000 .500 0

Q12_3 2 45 64 .625 .313 18

Q12_4 2 91 100 1.000 .500 9

Q13 2 55 82 .250 .125 27

Q14 4 0 9 1.000 .500 9

Q15 4 64 91 .250 .125 27

Q16 1 91 91 1.000 .500 0

Q17 1 100 91 1.000 .500 -9

Q18 1 100 100 na na 0

Q19 2 64 91 .250 .125 27

Q20 2 100 91 1.000 .500 -9

273

Q21 2 91 91 1.000 .500 0

Q22 3 64 82 .625 .313 18

Q23_1 3 91 100 1.000 .500 9

Q23_2 3 91 73 .625 .313 -18

Q23_3 3 82 82 1.000 .500 0

Q23_4 3 64 82 .625 .313 18

Q23_5 3 91 100 1.000 .500 9

Q23_6 3 82 100 .500 .250 18

Q23_7 3 82 100 .500 .250 18

Q24 4 82 100 .500 .250 18

Q25 4 91 100 1.000 .500 9

Q26 1 100 100 na na 0

Q27 4 82 100 .500 .250 18

Q28 4 91 91 1.000 .500 0

Q29 4 91 100 1.000 .500 9

Q30 4 82 91 1.000 .500 9

Q31 4 100 100 na na 0

Q32_1 3 100 100 na na 0

Q32_2 3 100 100 na na 0

Q32_3 3 100 91 1.000 .500 -9

Q32_4 3 82 91 1.000 .500 9

Q32_5 3 82 100 .500 .250 18

Q32_6 3 100 100 na na 0

Q33_1 2 100 100 na na 0

Q33_2 2 100 100 na na 0

Q33_3 2 91 91 1.000 .500 0

Q33_4 2 100 91 1.000 .500 -9

Q33_5 2 91 91 1.000 .500 0

Q33_6 2 100 100 na na 0

274

Note: EoL Care 2005/2006 Items with Positive Pre-/Posttest Difference

275

Note: EoL Care 2005/2006 Items with No or Negative Difference from Pre- to Posttest

KNOWLEDGE BUILDING IN CONTINUING MEDICAL ...

Documents

Transcript of KNOWLEDGE BUILDING IN CONTINUING MEDICAL ...