KNOWLEDGE BUILDING IN CONTINUING MEDICAL EDUCATION
by
Leila Rachel Lax
A thesis submitted in conformity with the requirements
for the degree of Doctor of Philosophy
Department of Theory and Policy Studies in Education
Ontario Institute for Studies in Education
University of Toronto
© Copyright by Leila Rachel Lax 2012
ii
KNOWLEDGE BUILDING IN CONTINUING MEDICAL EDUCATION
Doctor of Philosophy, 2012
Leila Rachel Lax
Department of Theory and Policy Studies in Education
University of Toronto
ABSTRACT
Continuing medical education has been characterized as didactic and ineffective. This thesis
explores the use of Knowledge Building theory, pedagogy, and technology to test an alternative
model for physician engagement—one that emphasizes sustained and creative work with ideas.
Several important conceptual changes in continuing medical education are implied by the
Knowledge Building model—changes that extend the traditional approach through engagement
in (a) collective responsibility for group achievements rather than exclusive focus on individual
advancement and (b) work in design-mode, with ideas treated as objects of creation and
assemblage into larger wholes and new applications, with extension beyond belief-mode where
evidence-based acceptance or rejection of beliefs dominates. The goal is to engage physicians in
“cultures of participation” where individual learning and collective knowledge invention or
metadesign advance in parallel.
This study was conducted in a continuing medical education End-of-Life Care Distance
Education course, for family physicians, from 2004 to 2009. A mixed methods case study
methodology was used to determine if social-mediated Knowledge Building improved
physicians’ knowledge, and if so, what social network structural relationships and sociocognitive
dynamics support knowledge improvement, democratization of knowledge, and a metadesign
perspective.
iii
Traditional pre-/posttest learning measures across 4-years showed significant gains (9% on
paired t-test = 5.34, p < 0.001) and large effect size (0.82). Social network analysis of ten
2008/2009 modules showed significant difference in density of build-on notes across groups.
Additional results demonstrated a relationship between high knowledge gains and social network
measures of centrality/distribution and cohesion. Correlation of posttest scores with centrality
variables were all positive. Position/power analyses highlighted core-periphery sociocognitive
dynamics between the facilitator and students. Facilitators most often evoked partner/expert
relationships. Questions rather than statements dominated the discourse; discourse complexity
was elaborated/compiled as opposed to reduced/dispersed. Themes beyond predefined learning
objectives emerged and Knowledge Building principles of community responsibility, idea
improvability, and democratization of knowledge were evident. Overall, results demonstrate the
potential of collective Knowledge Building and design-mode work in continuing medical
education, with individual learning representing an important by-product. There were no
discernible decrements in performance, suggesting significant advantages rather than tradeoffs
from engagement in Knowledge Building.
iv
ACKNOWLEDGEMENTS
I would like to express my sincere gratitude to my thesis supervisor and mentor, Dr. Marlene
Scardamalia, who has forever changed the way I think, work, and look at the world, and inspired
my desire to “advance the edge.” I am also most grateful to the members of my thesis committee,
Dr. Michael Skolnik, Dr. Jim Hewitt, and Dr. Judy Watt-Watson, who have been insightful
teachers and wonderful mentors throughout my years at OISE, and my external appraiser, Dr.
Beatrice Ligorio. I would also like to acknowledge my fellow researchers at the Institute of
Knowledge Innovation and Technology at OISE/UT and the international community of the
Knowledge Society Network. I extend sincere thanks to my external social network analyst, Dr.
Don Philip, statistician Ms. Susan Elgie, and editors Susana Larosa and Mary Anne Carswell. In
addition, I thank the many academic/clinicians, professors, educational researchers, and
simulators I have had the opportunity to work with throughout the years, at the Wilson Centre for
Research in Medical Education, the Faculty of Medicine, the University of Toronto Centre for
the Study of Pain, the Faculty of Nursing, and the Standardized Patient Program, who have
become valued colleagues throughout the years, especially Dr. Judy Watt-Watson, Dr. Michael
McGillion, Dr. Lynn Russell, Dr. Cathy Smith, Laura Jayne Nelles, Dr. Niall Byrne, and Dr.
Glenn Regehr. The End-of-Life Care Distance Education Program could not have been
developed without the vision of Dr. Lawrence Librach. I thank my extraordinary collaborator,
Dr. Anita Singh, for her medical expertise in palliative care and contributions to the creation,
implementation, and ongoing success of the End-of-Life Care Distance Education Program. I am
indebted to the palliative care specialists, Dr. Anoo Tamber, Dr. Hyon Kim, and Dr. Paolo
Mazzotta for their dedication to facilitating online Knowledge Building in this program and to
Ms. Nancy Bush, for her excellent administrative and organizational skills. I gratefully
acknowledge the former directors of the Faculty of Medicine Discovery Common, Dr. Lawrence
Spero and Dr. Avi Hyman, and media specialists Prof. Meaghan Brierley, Jenn Tse, and Ju Ho
Park, for their support and contribution to media development. And last but certainly not least, I
sincerely thank my colleagues in Biomedical Communications, especially Dr. Linda Wilson-
Pauwels, Prof. Nicholas Woolridge, Prof. Emerita Margot Mackay, and Prof. Emerita Nancy
Joy, and my students whose dedication to improvable ideas has been a constant inspiration.
v
TABLE OF CONTENTS
Abstract ........................................................................................................................................... ii
Acknowledgements ........................................................................................................................ iv
List of Tables ................................................................................................................................. xi
List of Figures .............................................................................................................................. xiv
Dedication .................................................................................................................................. xviii
CHAPTER 1: INTRODUCTION ....................................................................................................1
1.1 Overview ..............................................................................................................................1
1.2 Context .................................................................................................................................2
1.3 Theoretical Foundations.......................................................................................................4
1.4 Background ..........................................................................................................................7
1.5 Purpose ...............................................................................................................................10
1.6 Significance........................................................................................................................11
1.7 Research Questions ............................................................................................................12
1.7.1 Research Clusters and Subquestions ......................................................................12
1.8 Dissertation Organization ..................................................................................................14
CHAPTER 2: LITERATURE REVIEW .......................................................................................17
2.1 Introduction ........................................................................................................................17
2.2 Cluster 1: Traditional Conceptualizations of Continuing Medical Education ...................18
2.2.1 Continuing Medical Education for Individual Lifelong Learning .........................18
2.2.1.1 Definition ......................................................................................................18
2.2.1.2 From individual to socially mediated participatory metadesign ...................20
2.3 Cluster 2: Performance Over and Above Traditional Individual Outcomes, Through
Collective Knowledge Building .........................................................................................23
2.3.1 Knowledge Building Research in Medical and Health Sciences Education ..........23
2.3.2 Distinction Between Learning and Knowledge Building ......................................25
2.3.3 Historical and Cultural Relevance of Knowledge Building ..................................27
2.3.4 Belief Mode and Design Mode ..............................................................................31
2.3.5 The Knowledge Building Problem Space ..............................................................32
2.3.6. Participant Structures and the Codesign of Knowledge Building .........................33
2.3.6.1 Facilitating Knowledge Building ..................................................................33
2.3.6.2 Facilitating knowledge building in relationship to coaching
reflection-in-action ........................................................................................34
2.3.6.3 Expertise and social networks for cocreation ...............................................37
2.3.7 Knowledge Forum Suite of Analytic and Social Network Tools
to Support Knowledge Building ............................................................................38
2.4 Cluster 3: Social Network Measures and Sociocognitive Dynamics
That Enable Work Over and Above Traditional Learning ................................................40
2.4.1 Social Network Analysis in the Social Sciences....................................................40
2.4.2 Social Network Analysis and Power Structures in Education ...............................42
2.4.2.1 Power structures concepts .............................................................................42
2.4.2.2 Social network analysis of roles and power structures .................................44
2.4.2.3 Social network position and identity.............................................................44
2.4.2.4 Social network analysis of facilitator roles and participant structures .........45
2.4.3 Social Network Analysis of Sociocognitive Dynamics in
Knowledge Building ..............................................................................................46
vi
2.4.3.1 Social network analysis of Knowledge Building in K–12 classrooms .........46
2.4.3.2 Social network analysis of knowledge building in higher education ...........47
2.4.3.3 Social network analysis of an international knowledge building network ...48
2.4.4 Social Network Analysis in Medicine and Medical Education .............................49
2.4.5 Complexity of Social Network Discourse: Visual Representation,
Elaboration, and Identification of Misconceptions ................................................50
2.4.6 Analogies and Emergent Ideas in Abductive and Adductive Processes ................55
2.5 Summary and Implications ................................................................................................57
CHAPTER 3: METHODS .............................................................................................................59
3.1 Introduction ........................................................................................................................59
3.2 Context and Program Overview.........................................................................................59
3.3 Research Goal and Question ..............................................................................................60
3.4 Case Study Methodology ...................................................................................................61
3.5 Pedagogic Design...............................................................................................................62
3.5.1 Multimedia Design of Knowledge Forum Communal Space ................................63
3.5.2 Knowledge Building Theory-Based Design ..........................................................64
3.6 Research Methods ..............................................................................................................66
3.6.1 Participants .............................................................................................................67
3.6.2 Instrumentation and Procedures .............................................................................67
3.6.2.1 Pain knowledge pre- and posttests ................................................................68
3.6.2.2 Attitudes and opinions survey .......................................................................69
3.6.2.3 Knowledge Forum online activity and interactivity .....................................70
3.6.3 Data Collection and Analyses: Pilot Study 2004/2005 ..........................................70
3.6.4 Data Collection and Analyses Cluster 1: Traditional Measures
Across Years 2005/2009 ........................................................................................70
3.6.4.1 Analyses of pain knowledge improvement ...................................................70
3.6.4.2 Attitudes and opinions of collaborative online work ....................................70
3.6.5 Data Collection and Analyses Cluster 2: Activity, Interactivity,
and Social Network Measures Over and Above Traditional Measures .................71
3.6.5.1 Knowledge Forum online activity and interactivity .....................................71
3.6.5.2 Knowledge Forum graphic contribution and social network analyses .........71
3.6.5.3 2-way ANOVA .............................................................................................71
3.6.5.4 Social network structural analyses ................................................................71
3.6.5.5 Significant difference and effect size of social network density and
centrality measures between groups, with and without facilitator ................72
3.6.5.6 Relationship of social network structural analyses of the three
pain modules to pain pre-/posttest scores .....................................................72
3.6.6. Data Collection and Analyses Cluster 3: Social Network Analyses and
Measures of Sociocognitive Dynamics That Support Knowledge Building
Outcomes, Over and Above Learning ...................................................................72
3.6.6.1 Social network position and power analyses ................................................72
3.6.6.2 Social network content analyses ...................................................................73
3.6.6.2.1 Social network analysis of facilitator and student patterns
of discourse ..........................................................................................73
3.6.6.2.2 Content analysis of themes of discourse, beyond learning
objectives .............................................................................................73
3.6.6.2.3 Complexity of clinical discourse .........................................................74
vii
3.6.6.2.4 Evidence of Knowledge Building indicators and exemplars ...............74
3.6.7 Summary ................................................................................................................74
CHAPTER 4: RESULTS—PILOT STUDY .................................................................................78
4.1 Introduction ........................................................................................................................78
4.2 Pilot Study, 2004/2005: Results .........................................................................................78
4.2.1 Pain Knowledge, 2004/2005: Pre- and Posttest Results ........................................78
4.2.2 Pilot Study, 2004/2005: Attitudes and Opinions Survey Results ..........................80
4.2.3 Pilot Study, 2004/2005: Online Activity Analytic Toolkit Measures....................82
4.2.4 Pilot Study, 2004/2005: Summary and Iterative Design Recommendations .........83
CHAPTER 5: RESULTS—CLUSTER 1: TRADITIONAL OUTCOME MEASURES ..............85
5.1 Introduction ........................................................................................................................85
5.2 Pain Knowledge Pre-/Posttest Results by Year and Cumulatively Across Four Years .....86
5.2.1 Pain Knowledge 2005/2006 Pre-/Posttest Matched Results ..................................87
5.2.2 Pain Knowledge 2006/2007 Pre-/Posttest Matched Results ..................................88
5.2.3 Pain Knowledge 2007/2008 Pre-/Posttest Matched Results ..................................89
5.2.4 Pain Knowledge 2008/2009 Pre-/Posttest Matched Results ..................................89
5.2.5 Cumulative Matched Results of 2005–2009 Pain Knowledge Pre-/Posttests .......90
5.2.6 Summary of Results of Pain Knowledge Pre-/Posttests ........................................90
5.3 Attitude and Opinion Survey Results 2005–2009 .............................................................91
5.4 Summary of Traditional Outcome Measures: 2005–2009 Results of Pain
Knowledge Pre-/Posttest Results and Attitude and Opinion Survey Results ....................96
CHAPTER 6: RESULTS—CLUSTER 2: PERFORMANCE OVER AND ABOVE
TRADITIONAL MEASURES .................................................................................................97
6.1 Introduction ........................................................................................................................97
6.2 Online Activity and Interactivity Measures Results ..........................................................98
6.2.1 Results of 2005–2009 Read, Write, Build-On Measures .......................................99
6.3 Results of 2008/2009 Online Performance and Social Network Relationship
Measures (Beyond Learning) ...........................................................................................102
6.3.1 Group 1, 2008/2009 ATK and Social Network Assessment Measures ...............103
6.3.1.1 Group 1, Mr. Singh’s Pain, Part 1 ...............................................................103
6.3.1.2 Group 1, Mr. Singh’s Pain, Part 2 ...............................................................106
6.3.1.3 Group 1, Mary’s Misery, Pain Module 3 ....................................................109
6.3.1.4 Group 1, Judy’s Last Days, Part 1 ..............................................................112
6.3.1.5 Group 1, Judy’s Last Days, Part 2 ..............................................................115
6.3.2 Group 2, 2008/2009 ATK and Social Network Assessment Measures ...............118
6.3.2.1 Group 2, Mr. Singh’s Pain, Part 1 ...............................................................118
6.3.2.2 Group 2, Mr. Singh’s Pain, Part 2 ...............................................................121
6.3.2.3 Group 2, Mary’s Misery, Pain Module 3, Group 2 .....................................124
6.3.2.4 Judy’s Last Days, Part 2, Group 2 ..............................................................127
6.3.2.5 Judy’s Last Days, Part 2, Group 2 ..............................................................130
6.3.3 Social Network Pattern Analysis Across Modules and Comparatively
Between Groups, 2008/2009 ................................................................................133
6.3.4 Summary of Social Network Analysis and Knowledge Forum Tools .................138
6.4 Results for Groups 1 and 2 of 2-Way ANOVA of 2008/2009 Pain Knowledge
Pre-/Posttests ....................................................................................................................139
viii
6.5 Social Network Structural Analysis Results 2008/2009 ..................................................141
6.5.1 Structural SNA: Network Links and Network Density Results...........................142
6.5.1.1 Network density results of t-tests and effect ...............................................144
6.5.2 Social Network: Eigenvector, In-degree, and Out-degree Centrality Results .....145
6.5.2.1 t-test and effect size results of social network centrality ............................147
6.5.2.2 t-test and effect size results of social network density and centrality
measures across groups ...............................................................................149
6.6 Social Network Analyses Results of Cliques (K) and Cohesion Index of Build-On
Ideas Only ........................................................................................................................150
6.7 Relationship of Social Networks to Knowledge Improvement Scores ............................153
6.7.1 Relationship of Social Network Structural Analyses of the Three Pain
Modules to Pain Knowledge Improvement Pre-/Posttest Scores ........................153
6.7.2 Correlation Between Results of Social Network Analyses and Pain
Posttest Scores .....................................................................................................155
6.8 Cluster 2 Summary of Performance Measures (Over and Above
Traditional Learning) From Social Network Structural Results ......................................156
CHAPTER 7: RESULTS—CLUSTER 3: SOCIAL NETWORK POSITION/POWER
ANALYSIS AND SOCIOCOGNITIVE DYNAMICS OF KNOWLEDGE BUILDING .....159
7.1 Introduction ......................................................................................................................159
7.2 Results of Social Network Position/Power Analysis .......................................................161
7.2.1 Relationship of Individual Student Social Network Position Scores
to Difference on Pre-/Posttest Knowledge Scores ...............................................161
7.2.2 Social Network Position, and Power Maps and Measures ..................................162
7.2.2.1 Position/power results for Group 1, Mr. Singh’s Pain, Part 1 ....................163
7.2.2.2 Position/power results for Group 1, Mr. Singh’s Pain, Part 2 ....................165
7.2.2.3 Position/power results for Group 1, Mary’s Misery ...................................167
7.2.2.4 Position/power results for Group 1, Judy’s Last Days, Part 1 ....................169
7.2.2.5 Position/power results for Group 1, Judy’s Last Days, Part 2 ....................171
7.2.2.6 Position/power analysis interpretation and conclusions for Group 1 .........174
7.2.2.7 Position/power results for Group 2, Mr. Singh’s Pain, Part 1 ....................175
7.2.2.8 Position/power results for Group 2, Mr. Singh’s Pain, Part 2 ....................176
7.2.2.9 Position/power results for Group 2, Mary’s Misery ...................................178
7.2.2.10 Position/power results for Group 2, Judy’s Last Days, Part 1 ....................180
7.2.2.11 Position/power results for Group 2, Judy’s Last Days, Part 2 ....................182
7.2.2.12 Position/power analysis interpretation and conclusion for Group 2 ...........185
7.3 Results of Social Network Content Analyses ..................................................................186
7.3.1 Social Network Analysis of Facilitator/Students Patterns of Discourse ..............187
7.3.1.1 Patterns of facilitator discourse ...................................................................187
7.3.1.2 Patterns of discourse statements and questions ..........................................188
7.3.2 Content Analysis of Themes, Beyond the Predetermined
Learning Objectives .............................................................................................191
7.3.2.1 Content analysis of knowledge work ..........................................................191
7.3.2.2 Relationship of themes, threads, and learning objectives in the social
network discourse .......................................................................................192
7.3.2.3 Summary of emergent themes and metadesign results ...............................195
7.3.3 Social Network Analysis of Complexity of Discourse ........................................196
ix
7.3.4 Evidence of Knowledge Building Indicators Within Social
Network Discourse...............................................................................................198
7.3.5 Cluster 3 Summary of Social Network Position and Power Analysis,
Sociocognitive Dynamics, and Indicators of Knowledge Building .....................210
7.3.5.1.1 Facilitation patterns ............................................................................211
7.3.5.1.2 Belief-mode and design-mode knowledge work and its
relationship to predefined learning objectives ...................................211
7.3.5.1.3 Emergent themes and metadesign (beyond predefined learning
objectives) ..........................................................................................211
7.3.5.1.4 Results of Knowledge Building indicators within social
network discourse ..............................................................................211
7.4 Overview of Results .........................................................................................................212
7.4.1 Pilot Study 2004/2005..........................................................................................212
7.4.2 Cluster 1: Traditional Measures ...........................................................................212
7.4.2.1 Matched results of pain knowledge pre-/posttests, 2005–2009 ..................212
7.4.2.2 Attitude and opinion results 2005–2009 .....................................................213
7.4.3 Cluster 2: Performance Over and Above Traditional Measures ..........................213
7.4.3.1 ATK Measures 2005–2009 .........................................................................213
7.4.3.2 Social network results (KF analytic tools) 2008/2009: Groups 1 and 2 .....214
7.4.3.2.1 Mr. Singh’s Pain, Part 1: Group 1 .....................................................214
7.4.3.2.2 Mr. Singh’s Pain, Part 2: Group 1 .....................................................214
7.4.3.2.3 Mary’s Misery: Group 1 ....................................................................214
7.4.3.2.4 Judy’s Last Days, Part 1: Group 1 .....................................................215
7.4.3.2.5 Judy’s Last Days, Part 2: Group 1 .....................................................215
7.4.3.2.6 Mr. Singh’s Pain, Part 1: Group 2 .....................................................215
7.4.3.2.7 Mr. Singh’s Pain, Part 2: Group 2 .....................................................215
7.4.3.2.8 Mary’s Misery: Group 2 ....................................................................215
7.4.3.2.9 Judy’s Last Days, Part 1: Group 2 .....................................................216
7.4.3.2.10 Judy’s Last Days, Part 2: Group 2 .....................................................216
7.4.3.2.11 Results of 2-way ANOVAs on 2008–2009 Groups 1 and 2 ..............216
7.4.3.2.12 Network links and density measures and tests of significant
difference and effect size ...................................................................216
7.4.3.2.13 Social network measures of centrality: Eigenvector, in-degree,
and out-degree; and results of t-test and effect size ...........................217
7.4.3.2.14 Significant difference between groups in social network density
measures .............................................................................................217
7.4.3.2.15 Clique members and clique cohesion index.......................................217
7.4.3.2.16 Relationship of SN structural analysis of three pain modules to
pre-/posttest knowledge improvement scores ....................................218
7.4.4 Cluster 3: Social Network Analyses and Sociocognitive Dynamics That
Support Knowledge Building Over and Above Learning ....................................218
7.4.4.1 Results of social network position/power analysis 2008/2009 ...................218
7.4.4.2 Results of social network content analysis 2008/2009 ...............................218
7.4.4.2.1 Facilitator/students patterns of discourse patterns .............................219
7.4.4.2.2 Results of analysis of complexity of discourse ..................................219
7.4.4.2.3 Emergent themes/threads in knowledge work (beyond
predefined learning objectives) ..........................................................219
x
7.4.4.2.4 Results of Knowledge Building indicators within social network
discourse ............................................................................................219
CHAPTER 8: DISCUSSION .......................................................................................................221
8.1 Introduction ......................................................................................................................221
8.2 Summary of Research ......................................................................................................221
8.3 Significance of Cluster 1 Research Results: Traditional Learning Outcomes
and the Relationship to Continuing Medical Education, Traditionally Conceived .........225
8.4 Significance of Cluster 2 Research Results: Social Network Performance
Measures Over and Above Learning ...............................................................................226
8.5 Significance of Cluster 3 Research Results: Sociocognitive Dynamics
That Enable Work Over and Above Learning .................................................................229
8.5.1 Facilitator/Participant Sociocognitive Dynamics ................................................229
8.5.2 Belief- and Design-Mode Knowledge Work .......................................................230
8.5.3 Knowledge Building Principles and Metadesign Concepts .................................231
8.5.3 Collective Responsibility and Democratizing Knowledge ..................................232
8.6 Summary ..........................................................................................................................236
CHAPTER 9: CONCLUSIONS ..................................................................................................237
9.1 Introduction ......................................................................................................................237
9.2 Research Questions and Answers ....................................................................................237
9.3 Strengths of This Research Study ....................................................................................239
9.4 Limitations .......................................................................................................................240
9.5 Future Research ...............................................................................................................240
9.6 Final Remarks ..................................................................................................................241
REFERENCES ............................................................................................................................243
APPENDICES .............................................................................................................................263
Appendix A: Information Letter and Informed Consent Form ....................................................264
Appendix B: End-of-Life Care Distance Education Program Homepage and Schedule .............266
Appendix C: End-of-Life Care Distance Education Program Modules in
Knowledge Forum® and Multimedia Case Note ....................................................................267
Appendix D: Pain Pre-/Posttest Item Analysis 2004/2005 ..........................................................268
Appendix E: Pain Pre-/Posttest Item Analysis 2005/2006 ..........................................................272
xi
LIST OF TABLES
Table 1. Research Clusters and Organization of Dissertation .....................................................16
Table 2. End-of-Life Care Distance Education Program Participants and Data Collection
Instruments ................................................................................................................69
Table 3. Demographics of the Study Population .........................................................................75
Table 4. Research Question/Subquestions, Data Analysis, and Data Sources ............................75
Table 5. Matched Results From the EoL Care Pain Pre-/Posttests, 2004/2005 ...........................79
Table 6. Responses to Attitudes and Opinions Survey, 2004/2005 (n = 10 of 13) ......................81
Table 7. Online Activity Measures by Group, 2004/2005 ...........................................................82
Table 8. Research Question .........................................................................................................85
Table 9. Cluster 1: Traditional Measures .....................................................................................86
Table 10. Matched Results of Pain Pre-/Posttests, 2005/2006 ......................................................87
Table 11. Matched Results of Pain Pre-/Posttests, 2006/2007 ......................................................88
Table 12. Matched Results of the 2007/2008 Pain Pre-/Posttests .................................................89
Table 13. Matched Results of the 2008/2009 Pain Pre-/Posttests .................................................89
Table 14. Cumulative Matched Results of the 2005–2009 Pain Pre-/Posttests .............................90
Table 15. Percentaged Results of Attitude and Opinions Summative Survey, 2005–2009 ...........91
Table 16. Cluster 2: Beyond Traditional Measures .......................................................................97
Table 17. Online Activity Measures from 2005/2006, 2006/2007, and 2007/2008 ....................100
Table 18. Summary of Online Activity and Interactivity Measures, 2008/2009 .........................101
Table 19. Results of 2-way ANOVA Groups 1 and 2, 2008/2009 ..............................................140
Table 20. Number of Edges and Network Density of Build-on Notes With and Without
Facilitator, Group 1 .................................................................................................142
Table 21. Number of Edges (Links) and Network Density of Build-ons With and Without
Facilitator, Group 2 .................................................................................................143
Table 22. Summary of Edges/Links and Network Density: Groups 1 and 2 ...............................144
Table 23. Comparison of Density with and Without Facilitator of Notes
Built-On and Read ...................................................................................................145
Table 24. Social Network Centrality Measures (in Percentages) With and Without
Facilitator, Group 1 .................................................................................................146
Table 25. Social Network Centrality Measures (in Percentages) With and Without
Facilitator, Group 2 .................................................................................................147
Table 26. Comparisons of SN Centrality Measures With and Without Facilitator,
by Group, Across All Modules ................................................................................148
Table 27. Comparisons of SN Measures, between Groups 1 and 2, With Facilitator,
Across All Modules .................................................................................................149
xii
Table 28. Clique Members and Cohesion Index Results of Build-Ons by Group
With Facilitator ........................................................................................................151
Table 29. Clique and Cohesion Index Results of Build-On Notes in Groups1 and 2,
With Facilitators ......................................................................................................152
Table 30. Comparative Summary of SN Results of Groups 1 and 2 in Three Pain Modules......154
Table 31. Comparisons of Social Network Measures of Build-Ons, Between Groups,
Across Three Pain Modules .....................................................................................154
Table 32. Results of 2008/2009 Pain Pre-/Posttests ....................................................................155
Table 33. Spearman Correlations of the Post Score with Social Network Variables ..................156
Table 34. Cluster 3: Social Network Analysis of Sociocognitive Dynamics ..............................160
Table 35. Group 1 Individual Differences in Student Pain Knowledge Pre-/Posttest Scores .....161
Table 36. Group 2 Individual Differences in Student Pain Knowledge Pre-/Posttest Scores .....162
Table 37. Social Network Position/Power Map and Centrality Measures for Group 1,
Mr. Singh’s Pain, Part 1, Build-On Notes ...............................................................164
Table 38. Social Network Position/Power Map and Centrality Measures for Group 1,
Mr. Singh’s Pain, Part 2, Build-On Notes ...............................................................166
Table 39. Social Network Position/Power Map and Centrality Measures for Group 1,
Mary’s Misery, Build-On Notes ..............................................................................168
Table 40. Social Network Position/Power Map and Centrality Measures for Group 1,
Judy’s Last Days, Part 1, Build-On Notes ...............................................................170
Table 41. Social Network Position/Power Map and Centrality Measures for Group 1,
Judy’s Last Days, Part 2, Build-On Notes ...............................................................172
Table 42. Social Network Position/Power Map and Centrality Measures for Group 2,
Mr. Singh’s Pain, Part 1, Build-On Notes ...............................................................175
Table 43. Social Network Position/Power Map and Centrality Measures for Group 2,
Mr. Singh’s Pain, Part 2, Build-On Notes ...............................................................177
Table 44. Social Network Position/Power Map and Centrality Measures for Group 2,
Mary’s Misery, Build-On Notes ..............................................................................179
Table 45. Social Network Position/Power Map and Centrality Measures for Group 2,
Judy’s Last Days, Part 1, Build-On Notes ...............................................................181
Table 46. Social Network Position/Power Map and Centrality Measures for Group 2,
Judy’s Last Days, Part 2, Build-On Notes ...............................................................183
Table 47. Facilitator/Participant Discourse Patterns: Discourse Stance ......................................187
Table 48. Patterns of Discourse Statements and Questions by Facilitators and Students,
and Student/Facilitator Patterns at the Core ............................................................189
Table 49. Results of Knowledge Work With Predefined Learning Objectives or
Emergent Ideas ........................................................................................................191
Table 50. Summary of Emergent Themes/Threads Beyond Learning Objectives ......................194
xiii
Table 51. Semantic Analysis Results of Complexity of Discourse .............................................198
Table 52. Knowledge Building Principles Demonstrated in the Discourse.................................199
Table 53. Clusters Summary of Research Questions and Results of Analyses ...........................222
xiv
LIST OF FIGURES
Figure 1. Total note contributions Group 1, Mr. Singh’s Pain, Part 1, 2008/2009 ......................104
Figure 2. Total number of build-on notes created Group 1, Mr. Singh’s Pain, Part 1,
2008/2009 ................................................................................................................104
Figure 3. Social network analysis of who built-on whose notes, Group 1, Mr. Singh’s Pain,
Part 1, 2008/2009 .....................................................................................................105
Figure 4. Total number of notes read, Group 1, Mr. Singh’s Pain, Part 1, 2008/2009 ................105
Figure 5. Social network analysis of who read whose notes, Group 1, Mr. Singh’s Pain,
Part 1, 2008/2009 .....................................................................................................106
Figure 6. Total note contributions, Group 1, Mr. Singh’s Pain, Part 2, 2008/2009 .....................107
Figure 7. Total number of build-on notes created Group 1, Mr. Singh’s Pain, Part 2,
2008/2009 ................................................................................................................107
Figure 8. Social network analysis of who built-on whose notes, Group 1, Mr. Singh’s Pain,
Part 2, 2008/2009 .....................................................................................................108
Figure 9. Total number of notes read, Group 1, Mr. Singh’s Pain, Part 2, 2008/2009 ................108
Figure 10. Social network analysis of who read whose notes, Group 1, Mr. Singh’s Pain,
Part 2, 2008/2009 .....................................................................................................109
Figure 11. Total note contributions, Group 1, Mary’s Misery, 2008/2009 .................................110
Figure 12. Build-on note contributions, Group 1, Mary’s Misery, 2008/2009 ............................110
Figure 13. Social network analysis of who built-on whose notes, Group 1, Mary’s Misery,
2008/2009 ................................................................................................................111
Figure 14. Total number of notes read, Group 1, Mary’s Misery, 2008/2009.............................111
Figure 15. Social network analysis of who read whose notes, Group 1, Mary’s Misery,
2008/2009 ................................................................................................................112
Figure 16. Total note contributions, Group 1, Judy’s Last Days, Part 1, 2008/2009 ..................113
Figure 17. Build-on note contributions, Group 1, Judy’s Last Days, Part 1, 2008/2009.............113
Figure 18. Social network analysis of who built-on whose notes, Group 1, Judy’s Last
Days, Part 1, 2008/2009 ..........................................................................................114
Figure 19. Total number of notes read, Group 1, Judy’s Last Days, Part 1, 2008/2009 .............114
Figure 20. Social network analysis of who read whose notes, Group 1, Judy’s Last
Days, Part 1, 2008/2009 ..........................................................................................115
Figure 21. Total note contributions, Group 1, Judy’s Last Days, Part 2, 2008/2009 ..................116
Figure 22. Build-on note contributions, Group 1, Judy’s Last Days, Part 2, 2008/2009.............116
Figure 23. Social network analysis of who built-on whose notes, Group 1, Judy’s Last
Days, Part 2, 2008/2009 ..........................................................................................117
Figure 24. Total number of notes read, Group 1, Judy’s Last Days, Part 2, 2008/2009 .............117
xv
Figure 25. Social network analysis of who read whose notes, Group 1, Judy’s Last
Days, Part 2, 2008/2009 ..........................................................................................118
Figure 26. Total note contributions, Group 2, Mr. Singh’s Pain, Part 1, 2008/2009 ...................119
Figure 27. Total number of build-on notes created Group 2, Mr. Singh’s Pain, Part 1,
2008/2009 ................................................................................................................119
Figure 28. Social network analysis of who built-on whose notes, Group 2, Mr. Singh’s
Pain, Part 1, 2008/2009 ...........................................................................................120
Figure 29. Total number of notes read, Group 2, Mr. Singh’s Pain, Part 1, 2008/2009 ..............120
Figure 30. Social network analysis of who read whose notes, Group 2, Mr. Singh’s Pain,
Part 1, 2008/2009 .....................................................................................................121
Figure 31. Total note contributions, Group 2, Mr. Singh’s Pain, Part 2, 2008/2009 ...................122
Figure 32. Total number of build-on notes created Group 2, Mr. Singh’s Pain, Part 2,
2008/2009 ................................................................................................................122
Figure 33. Social network analysis of who built-on whose notes, Group 2, Mr. Singh’s
Pain, Part 2, 2008/2009 ...........................................................................................123
Figure 34. Total number of notes read, Group 2, Mr. Singh’s Pain, Part 2, 2008/2009 ..............123
Figure 35. Social network analysis of who read whose notes, Group 2, Mr. Singh’s Pain,
Part 2, 2008/2009 .....................................................................................................124
Figure 36. Total note contributions, Group 2, Mary’s Misery, 2008/2009 .................................125
Figure 37. Build-on note contributions, Group 2, Mary’s Misery, 2008/2009 ............................125
Figure 38. Social network analysis of who built-on whose notes, Group 2, Mary’s Misery,
2008/2009 ................................................................................................................126
Figure 39. Total number of notes read, Group 2, Mary’s Misery, 2008/2009.............................126
Figure 40. Social network analysis of who read whose notes, Group 2, Mary’s Misery,
2008/2009 ................................................................................................................127
Figure 41. Total note contributions, Group 2, Judy’s Last Days, Part 1, 2008/2009 ..................128
Figure 42. Build-on note contributions, Group 2, Judy’s Last Days, Part 1, 2008/2009.............128
Figure 43. Social network analysis of who built-on whose notes, Group 2, Judy’s Last
Days, Part 1, 2008/2009 ..........................................................................................129
Figure 44. Total number of notes read, Group 2, Judy’s Last Days, Part 1, 2008/2009 .............129
Figure 45. Social network analysis of who read whose notes, Group 2, Judy’s Last Days,
Part 1, 2008/2009 .....................................................................................................130
Figure 46. Total note contributions, Group 2, Judy’s Last Days, Part 2, 2008/2009 ..................131
Figure 47. Build-on note contributions, Group 2, Judy’s Last Days, Part 2, 2008/2009.............131
Figure 48. Social network analysis of who built-on whose notes, Group 2, Judy’s Last
Days, Part 2, 2008/2009 ..........................................................................................132
Figure 49. Total number of notes read, Group 2, Judy’s Last Days, Part 2, 2008/2009 .............132
xvi
Figure 50. Social network analysis of who read whose notes, Group 2, Judy’s Last Days,
Part 2, 2008/2009 .....................................................................................................133
Figure 51. Group 1, Mr. Singh’s Pain, Part 1 ..............................................................................134
Figure 52. Group 1, Mr. Singh’s Pain, Part 2 ..............................................................................134
Figure 53. Group 1, Mary’s Misery .............................................................................................135
Figure 54. Group 1, Judy’s Last Days, Part 1 ..............................................................................135
Figure 55. Group 1, Judy’s Last Days, Part 2 ..............................................................................135
Figure 56. Group 2, Mr. Singh’s Pain, Part 1 ..............................................................................136
Figure 57. Group 2, Mr. Singh’s Pain, Part 2 ..............................................................................137
Figure 58. Group 2, Mary’s Misery .............................................................................................137
Figure 59. Group 2, Judy’s Last Days, Part 1 ..............................................................................137
Figure 60. Group 2, Judy’s Last Days, Part 2 ..............................................................................138
Figure 61. Social network position/power map Group 1, Mr. Singh, Part 1, Build-on notes. ....165
Figure 62. Social network position/power map Group 1, Mr. Singh, Part 2, Build-on notes. ....167
Figure 63. Social network position/power map Group 1, Mary’s Misery, Build-on notes. ........169
Figure 64. Social network position/power map Group 1, Judy’s Last Days, Part 1,
Build-ons. ................................................................................................................171
Figure 65. Social network position/power map Group 1, Judy’s Last Days, Part 2,
Build-ons. ................................................................................................................173
Figure 66. Social network position/power map Group 2, Mr. Singh’s Pain, Part 1,
Build-ons. ................................................................................................................176
Figure 67. Social network position/power map Group 2, Mr. Singh’s Pain, Part 2,
Build-ons. ................................................................................................................178
Figure 68. Social network position/power map Group 2, Mary’s Misery, Build-on notes. ........180
Figure 69. Social network position/power map Group 2, Judy’s Last Days, Part 1,
Build-ons. ................................................................................................................182
Figure 70. Social network position/power map Group 2, Judy’s Last Days, Part 2,
Build-ons. ................................................................................................................184
Figure 71. Exemplar 1: 2008/09, Group 1, Mr. Singh’s Pain, Part 1. .........................................200
Figure 72. Exemplar 2: 2008/09, Group 1, Mr. Singh’s Pain, Part 1. .........................................201
Figure 73. Exemplar 3: 2008/09, Group 1, Mr. Singh’s Pain, Part 1. .........................................201
Figure 74. Exemplar 4: Group 1, Judy’s Last Days, Part 2. ........................................................202
Figure 75. Exemplar 5: Group 1, Judy’s Last Days, Part 2. ........................................................203
Figure 76. Exemplar 6: Group 1, Judy’s Last Days, Part 2. ........................................................204
Figure 77. Exemplar 7: Group 1, Judy’s Last Days, Part 2. ........................................................204
Figure 78. Exemplar 8: Group 1, Judy’s Last Days, Part 2. ........................................................205
xvii
Figure 79. Exemplar 9: Group 1, Judy’s Last Days, Part 2. ........................................................205
Figure 80. Exemplar 10: Group 1, Judy’s Last Days, Part 2. ......................................................206
Figure 81. Exemplar 11: Group 1, Judy’s Last Days, Part 2. ......................................................206
Figure 82. Exemplar 12: Group 1, Judy’s Last Days, Part 2. ......................................................207
Figure 83. Exemplar 13: Group 1, Judy’s Last Days, Part 2. ......................................................207
Figure 84. Exemplar 14: Group 1, Judy’s Last Days, Part 2. ......................................................208
Figure 85. Exemplar 15: Group 1, Judy’s Last Days, Part 2. ......................................................208
Figure 86. Exemplar 16: Group 1, Judy’s Last Days, Part 2—Reflections .................................209
Figure 87. Exemplar 17: Group 1, Judy’s Last Days, Part 2—Reflections .................................209
Figure 88. Exemplar 18: Group 1, Judy’s Last Days, Part 2—Reflections .................................210
Figure 89. Exemplar 19: Group 1, Judy’s Last Days, Part 2—Reflections .................................210
xviii
DEDICATION
This study is dedicated to the memory of my parents, Irving and Bella Goldstein, and
with never-ending love to my husband, Gary, and my three children,
Ryan, Ilyse, and Isaac Lax.
Knowledge Building in Continuing Medical Education … 1
CHAPTER 1
INTRODUCTION
1.1 Overview
Continuing medical education has been characterized as didactic and ineffective (Abrahamson et
al., 1999; Davis, 2011; Davis et al., 1999; Mazmanian & Davis, 2002; Miller et al., 2008) and in
process of “transitioning from an instructor-centric to a learner-centric model” that includes a
necessary shift from “time-based to value-based” systems (Dorman & Miller, 2011, p. 1339;
Pisacane, 2008). This thesis explores the use of Knowledge Building theory, pedagogy, and
technology, as defined by Scardamalia and Bereiter (2003a, 2006), to test a different model for
physician engagement, that is broader and potentially more expansive—one that places emphasis
on sustained and creative work with ideas—yet is consistent with professional responsibilities of
life-long learning.
Several important conceptual changes in continuing medical education are implied by the
Knowledge Building model—changes that extend the traditional approach through engagement
in (a) collective responsibility for group achievements rather than exclusive focus on individual
advancement and (b) work in design-mode, with ideas treated as objects of creation and
assemblage into larger wholes and new applications, with extension beyond belief-mode where
evidence-based acceptance or rejection of beliefs dominates. The goal is to engage physicians in
“cultures of participation” where individual learning and collective knowledge invention, or
metadesign, advance in parallel (Fischer, 2010, p. 168). This type of knowledge work is aimed at
deliberate and sustained innovation.
The goal of this thesis is to show improvements according to both traditional and non-traditional
measures—to show that there is not tradeoffs but rather improvements on both fronts. This thesis
will describe (a) traditional learning outcomes based on individual measures, (b) performance
over and above traditional measures (i.e., beyond learning as traditionally conceived and
measured), and (c) sociocognitive dynamics that enable work over and above traditional
learning.
Knowledge Building in Continuing Medical Education … 2
Detailed examination of the final year of this study will describe sociocognitive dynamics
through analyses of social network relationships in a Knowledge Building community dedicated
to knowledge improvement. Results demonstrate how a Knowledge Building approach can lead
to more expansive knowledge gains, over and above those predefined by learning objectives in
the curriculum. Making explicit complex relationships embedded in social networks structures
that support a Knowledge Building community is intended to provide more robust understanding
of collective knowledge construction in continuing medical education, as well as corresponding
measures, indicators, and attributes of community engagement.
1.2 Context
This study was conducted in the context of a University of Toronto continuing medical education
course for family physicians, called the End-of-Life Care Distance Education Program. Analyses
were performed across five years of the program, between 2004 and 2009. In-depth comparative
analysis of the social network structures of two groups, consisting of 10 modules, in the last year
of the study (2008/2009), was also performed. In addition, 40% of the 2008/2009 within-note
discourse was analyzed to verify participant structures, determine thematic content, and assess
strength of Knowledge Building indicators. A mixed-methods, (case study methodology was
used to determine if socially mediated processes underlying Knowledge Building improved
physicians’ knowledge, and if so, what social network structure and power relationships
supported knowledge improvement, democratization of participation, and curriculum cocreation
in continuing medical education.
Many family physicians in current practice have not had the opportunity to study palliative care.
Until recently most Canadian and American schools did not include palliative and end-of-life
care in their curricula (Sullivan et al., 2004). Over the past decade or so, this issue has been
addressed in undergraduate, graduate, and continuing medical education.
Numerous papers (Singer & Bowman, 2002; Singer, Martin, & Kelner, 1999; Sullivan et al.,
2004; WHO, 1990), provided necessary background on the state of palliative care education and
led to a series of studies and reports directed toward effecting change and implementation.
Canada’s Romanow Commission (Romanow, 2002) recommended implementation of palliative
care education to support the growth of an aging Canadian population. Since palliative care, had
Knowledge Building in Continuing Medical Education … 3
not been formally taught in most undergraduate medical education curricula, funding was
provided to the provincial governments to address this issue, and the Framework for a National
Strategy on Palliative and End-of-Life Care (Quality End-of-Life Care Coalition of Canada,
2005) emerged.
The End-of-Life Care Distance Education Program was designed to meet this need and fill this
gap in continuing medical education for physicians in the Toronto, York, and Simcoe regions. It
was developed, implemented, and is sustained by funding from the Ontario Ministry of Health
and Long-Term Care, through the Temmy Latner Centre for Palliative Care, Mount Sinai
Hospital, Toronto, Ontario, Canada and offered through the University of Toronto, Faculty of
Medicine, Office of Continuing Education and Professional Development. The Program is
accredited by the College of Family Physicians of Canada Maintenance of Proficiency program
(Mainpro®
) for annual maximum continuing medical education credits (25 Mainpro credits).
Numerous other programs in palliative and end-of-life care have recently been implemented
across Canada and the United States and a variety of research reports from all levels of medical
education have been published about undergraduate medical education (Wear, 2002; Wood,
Meekin, Fins, & Fleischman, 2004), postgraduate residency (Liao, Alpesh, & Rucker, 2004;
Porter-Williamson, von Gunten, Arman, et al., 2004; Weissman & Block, 2002), and physicians’
practice (Bradley et al., 2004; Brennan, 2002). Recommendations on how to improve aspects of
palliative care, such as current pain treatments, patient/family communication, and
interdisciplinary care (Desa et al., 2008; Fineberg, Wenger, & Forrow, 2004; Morrison & Meier,
2004) are also evident in the literature. Many courses are offered in person and some are offered
through web-based learning (Pereira et al., 2008).
It is noteworthy that the online pedagogic design of the End-of-Life Care Distance Education
Program (End-of-Life Care Distance Education, 2004) used in this study was based on
Knowledge Building theory and employs supporting Knowledge Forum®
technology. The
educational design is framed by predefined learning objectives and opportunities for open-ended,
emergent discourse around participant-identified ideas and issues. The five online modules are
structured around palliative care cases; (media clinical scenarios, digital resources, evaluation
components are embedded within. A palliative care expert facilitates collective discourse
amongst the group of family physicians. Asynchronous discussion extends over a one-month
Knowledge Building in Continuing Medical Education … 4
time frame in each of the five online modules. More detailed description of the pedagogic design
of the program is described elsewhere (Lax, Singh, Scardamalia, & Librach, 2006). Participant
discourse and social network interactions in the End-of-Life Care Distance Education Program
are based on the physicians’ discourse captured in Knowledge Forum and analysis using tools
embedded in this environment.
1.3 Theoretical Foundations
The theoretical foundations of this study are based on Knowledge Building as elaborated by
Scardamalia and Bereiter (Scardamalia, 2003a; Scardamalia & Bereiter, 2006); on associated
principles (Scardamalia, 2002) and ideas of continually improvable expertise (Bereiter &
Scardamalia, 1993); and on Bereiter’s (2002c) Education and Mind in the Knowledge Age.
Throughout the thesis the term Knowledge Building is capitalized to denote this specific
approach. There are, of course, many other approaches to knowledge building and when the
phrase is used in a more general sense, it is not capitalized.
Knowledge Building theory (Scardamalia & Bereiter, 2003a) and Knowledge Forum technology
(Scardamalia & Bereiter, 2003b, 2006) support a sociocognitive model of knowledge creation.
Knowledge Building is defined as “the production and continual improvement of ideas of value
to a community, through means that increase the likelihood that what the community
accomplishes will be greater than the sum of individual contributions and part of broader cultural
efforts” (Scardamalia & Bereiter, 2003a, p. 1370). Knowledge Building has been framed in
terms of work in design mode, to draw attention to the importance of processes aimed at
knowledge creation and innovation (Bereiter & Scardamalia, 2003, 2007; Scardamalia &
Bereiter, 2005). It is framed by the notion of continual progressive improvement and the
potentiality of what could be—a world view that implies going beyond what is.
The focus of learning on individual knowledge, which has been characterized as what’s in one’s
head, is reframed in Knowledge Building communities in terms of collaborative participation,
collective knowledge-work with ideas and artifacts, out in the world (Bereiter & Scardamalia,
1996; Scardamalia, 1999; Scardamalia & Bereiter, 1994, 2003a). Scardamalia and Bereiter
(2003a) have explained:
Learning is an internal, unobservable process that results in changes of belief, attitude, or
skill. Knowledge building, by contrast, results in the creation or modification of public
Knowledge Building in Continuing Medical Education … 5
knowledge—knowledge that lives “in the world” and is available to be worked on and
used by other people. Of course creating public knowledge results in personal learning,
but so does practically all human activity. (p. 1371)
Knowledge and ideas are seen as not contained within the mind but as an artifact in public space
to be collectively and collaboratively worked on (Bereiter, 2002c; Bereiter & Scardamalia, 1996;
Popper, 1972). Public space for the identification and problematization of ideas has been called a
“problem space” (Newell, 1980, p. 693); for Knowledge Building this could be called a design
space. Knowledge Forum provides an open space for communities to work with knowledge
artifacts.
Unlike most courseware environments, Knowledge Forum enables the representation of
connections between ideas and the interconnectedness of ideas (Hewitt, 2001; Hewitt &
Scardamalia, 1998; Lax, Scardamalia, Watt-Watson, Hunter, & Bereiter, 2010; Scardamalia,
2003a, 2004a). Most courseware environments use conversational threads to link one idea to the
next; Knowledge Forum employs graphical concept map representationality and functionality to
link multiple ideas and rise-above (Scardamalia, 2002), to create meta (higher-level) views and
perspectives. Additionally, Knowledge Building/Knowledge Forum assessment tools go well
beyond courseware read/post participation statistics, to graphically represent interactivity and
interconnectedness, and describe change over time (Teplovs, 2010; Teplovs & Scardamalia,
2007). This combination of statistical and graphic representations of data, particularly networks
of interactivity, make an important contribution to research assessment and embedded
assessment for students. These representations allow the researcher explicit comparison of
information not previously relatable and augment intuitive design (Akin, 2001). Knowledge
Forum has been specifically designed to provide sociocognitive support for Knowledge Building.
Knowledge Building has been described in terms of 12 interconnected principles (Scardamalia,
2002). Sociocognitive and technological determinants are defined for each and are often used, as
in this study to evaluate indicators of Knowledge Building within the discourse. Some of the
principles integral to this research study are: real ideas, authentic problems; improvable ideas;
idea diversity; rise above; epistemic agency; community knowledge, collective responsibility;
democratizing knowledge; symmetric knowledge advancement; constructive use of authoritative
sources, and Knowledge Building discourse. A complete description of each principle can be
Knowledge Building in Continuing Medical Education … 6
found in Scardamalia’s (2002) paper, “Collective Cognitive Responsibility for the Advancement
of Knowledge.”
Two important features of Knowledge Building discourse are evident in the description of these
principles: emergence and intentionality (Scardamalia & Bereiter, 1991, 2006; Scardamalia,
Bereiter, McLean, Swallow, & Woodruff, 1989). Scardamalia and Bereiter (2005) indicated:
Learning and knowledge creation are both emergent processes, sufficiently similar to
suggest that they are the same process, attaining, different levels of result. To say that
they are emergent is to say the output of the process (an advance in personal
understanding, a new theory, a design innovation, etc.) is not a deducible result of the
inputs. Yet all that educators really have in hand to influence learning and development
are inputs and limited control over environmental conditions. …An educational science
for the Knowledge Age must, we believe, treat ideas as real things and treat minds-
whether individual or collective—as dynamic systems. (p. 36)
Connectionist models of learning and development have been used to explain the concept of
emergence as a dynamic, self-organizing system (Bereiter, 2002c). Thus, it is important to design
for emergence—to provide opportunities and open spaces for Knowledge Building, over and
above learning. Many educational environments, concerned only with transmission of facts and
didactic teaching, do not.
Emergent, design mode work is defined as iterative, creative, and reflective work on personally
meaningful and authentic problems (Bereiter & Scardamalia, 2003). Design mode characterizes
the work of engineers, architects, and scientists. It embodies the notion of collective work with
ideas at the edge of current understanding.
Design mode situates ideas at the centre of education for communal Knowledge Building
throughout formal education and lifelong innovativeness (Bereiter & Scardamalia, 2003a). In
this way, Knowledge Building can be seen as a pervasive approach that transcends in-school
learning, beyond formal education and into continuing education, professional development, and
lifelong improvement. Central to the notion of education of a knowledge-creating process is
better integration of belief mode (the mode of learning facts and internalizing beliefs presented
by others) and design mode (the mode of idea improvement, invention, and theorizing).
In this thesis, Knowledge Building is considered a metadesign theory and Knowledge Forum is a
metadesign environment. Metadesign is defined as “Cultures of participation (that) provide all
citizens with means to become cocreators of new ideas, knowledge, and artifacts in personally
Knowledge Building in Continuing Medical Education … 7
meaningful ways” (Fischer, 2010, p. 168). Knowledge Building, framed as metadesign for
continuing medical education, includes learning, but focuses on the cocreation of ideas,
improvement of knowledge, and participation in the design of the educational agenda.
Knowledge Building communities for continuing medical education have the potential to aim
beyond predefined learning objectives (keeping up to date with the new facts and the latest
information), and instead to focus on emergent issues relevant to the community and aimed at
knowledge creation, innovation, and improvement of real ideas and authentic problems (Bereiter
& Scardamalia, 1993a; Scardamalia, 2002; Scardamalia & Bereiter, 1996, 2005).
This thesis challenges current limitations of continuing medical education, framed as individual
learning, and tests the possibilities of sociocognitive work to extend the boundaries towards
democratization of participation, cocreation of ideas, and metadesign of knowledge work. The
next section provides background on the limitations of continuing medical education as
traditionally conceived.
1.4 Background
Conceptualizations of continuing medical education, locally and abroad, are undergoing
reconsideration (University of Toronto, 2011; Wentz, 2011). Documents highlight opportunities
for increased participation through enhanced individual and community efforts. However this
conceptualization stops short of the potentiality captured in the conceptualization a Knowledge
Building community, that goes beyond increasing participation toward deeper sociocognitive
engagement and a broader, more expansive commitment to collectively engaging in the creation
and design of knowledge work, above and over what is typically a predetermined curriculum in a
continuing medical education course. The potential to engage sociocognitively not only as a
participant but as a designer of one’s own continuing medical education experiences is an
important distinction, and the essence of democratization and metadesign Knowledge Building,
as conceived of in this study.
Lifelong learning is the cornerstone of continuing medical education (Abrahamson et al., 1999;
Appelbaum, 2002; Collins, 2009; Manning & DeBakey, 2011). However, recent reports—by the
Macy Foundation (Hager, Russell, & Fletcher, 2007); the Agency for Healthcare Research and
Quality (Marinopoulos et al., 2007); the Conjoint Committee on Continuing Medical Education
Knowledge Building in Continuing Medical Education … 8
(Miller et al., 2008); and more recently and locally, the University of Toronto’s Faculty of
Medicine, Office of Continuing Education and Professional Development’s strategic plan 2011–
2016 (University of Toronto, 2011)—concur: continuing medical education is in need of
substantive change. The Conjoint Committee on Continuing Medical Education (Miller et al.,
2008) has stated:
To provide the best care to patients, a physician must commit to lifelong learning, but
continuing education and evaluation systems in the United States typically require little
more than record of attendance for professional association memberships, hospital staff
privileges, or reregistration of a medical license. (p. 95)
These reports have criticized continuing medical education by pointing to a disconnect between
requirements of licensing bodies and real-world practice. They recommend that continuing
medical education evolve, from counting hours of participation to recognizing physician
achievements in knowledge, competence, and performance. The Conjoint Committee on
Continuing Medical Education recommended that medical boards should require valid and
reliable assessment of physicians’ learning needs, and in doing so, the boards should collaborate
with physician and continuing medical education communities to assure that continuing medical
education achieves maximal benefit for physicians and patients—to assure the discovery and use
of best practices for continual professional development and maintenance of competence (Miller
et al., 2008). Many reports recommend that research in continuing medical education should be
raised to a national priority.
The current study addresses the challenge of this new agenda by pushing it even further, focusing
not only on work with best-practice knowledge, but on intentional work to move beyond best-
practice knowledge (Scardamalia, 2002). Reframing continuing medical education within a
Knowledge Building approach requires investment over and above learning—an intentionally
directed toward at sustained improvement of ideas, expertise, and patient care— that has the
potential to elevate community knowledge as well as individual achievements.
Reframing continuing medical education as continually improvable Knowledge Building
(Scardamalia & Bereiter, 2003a; Bereiter, 2002), may also require changes in attitude and
perceptions (Guest, Regehr, & Tiberius, 2001; Mylopoulos & Scardamalia, 2008). Moving from
a highly individualized culture primarily based on self-evaluation (Dunikowski, 2011; Campbell
& Parboosingh, 2011) towards a culture of community participation in sociocognitive knowledge
Knowledge Building in Continuing Medical Education … 9
work, and metadesign of curriculum (Fischer, 2009a, 2009b, 2010; Fischer & Giaccardi, 2006;
Fischer & Konomi, 2007; OECD, 2007), may require a deeper and broader commitment to
maintenance of competence (Brennan, 2002; Collins, 2009).
Physician competence is currently defined by the Royal College of Physicians and Surgeons of
Canada in the CanMEDS roles framework (Frank, 2005, p. 3), which puts the medical expert at
the centre, surrounded by the roles of scholar, health advocate, manager, collaborator,
communicator, and professional. The role of scholar emphasizes “four critical concepts: lifelong
learning/CPD, critical appraisal, research literacy, and teaching others” (Frank, 2005, p. 8). It
makes explicit the inclusion of teaching and research and the “ethical obligations for lifelong
maintenance of competence” (Frank, 2005, p. 8).
Nowhere in the CanMEDS framework is there mention of responsibility toward improvement of
knowledge artifacts of value to the community, and responsibility to work toward improvement
of the profession, as defined by pervasive Knowledge Building (Scardamalia, 2002; Scardamalia
& Bereiter, 2003a). Continuing medical education is thus conceived of as a one-way enterprise,
from transfer from those who know to those who don’t; as learning. There seems to be little or
no commitment or responsibility to knowledge improvement, beyond personal acquisition. The
potential for Knowledge Building is yet unrealized in any broad way in continuing medical
education. Knowledge Building democratization of continuing medical education, involving
opportunities for metadesign, cocreation, and collective innovation may require coevolution of
physician competencies, as learners and design agents in the advancement of their profession.
The present study will deal with the potential of Knowledge Building theory and Knowledge
Forum technology to support collective knowledge improvement in continuing medical
education, over and above (and yet including) individual learning. Three important contributions
a Knowledge Building theory and technology can make are: (a) support of sustained Knowledge
Building discourse, which involves cocreation of ideas, linkages between ideas, and movement
of discourse toward higher-level organization of ideas; (b) democratization of ideas, as
demonstrated by centrality of all participant ideas and opportunities for emergent knowledge
work as identified by the community; and (c) new measures of participation based on Knowledge
Building activity, interactivity, and social network relationships, which identify outcomes over
and above individual test scores, and help communities judge their own knowledge
Knowledge Building in Continuing Medical Education … 10
improvements, set goals, and design continuously improvable Knowledge Building
opportunities. The goal of this study is to examine the potential of Knowledge Building in
continuing medical education.
1.5 Purpose
The purpose of this study is to show improvements according to traditional as well as
nontraditional measures—to show that there can be improvements, not tradeoffs, on both fronts.
This thesis will describe (a) traditional learning outcomes based on individual measures; (b)
performance over and above traditional measures (i.e., beyond learning as traditionally
conceived and measured); and (c) sociocognitive dynamics that enable work over and above
traditional learning.
Currently, the structure and paradigm of continuing medical education have sustained some
criticism resulting in a series of recommendations for change that support higher levels of
individual participation and social engagement. This study builds-on recommendations for
greater opportunities for collaborative learning and more online initiatives in continuing medical
education.
Knowledge Building democratization of participation, design, and innovation is proposed. It is
important for physicians practising palliative care to better understand total pain management—
the scientific facts, the medical issues, and the mathematics of titration. However, it is also
important to provide family physicians with different levels of knowledge and experience in
palliative care, with opportunities to identify knowledge lacks, voice their concerns, professional
practice questions, patient problems, and ideas for potential improvement. By providing an open
problem-space, it is posited that participants in the course will become metadesigners, cocreators
of the content, the ideas, the knowledge, and the discourse. Thus, it is anticipated that each year
and in each group the curriculum (the ideas at the centre of the discourse space) will be
somewhat different, depending on the needs, interests, and directions that the participants want to
go in—as identified by the Knowledge Building community as important to them.
This thesis will describe characteristics and relationships in a social network deemed to be
successful by traditional measures, through pre-/posttests, through Knowledge Building
activity/interactivity, and through social network structural and content analyses.
Knowledge Building in Continuing Medical Education … 11
Social network analysis enables descriptions of structural relationships, based on network
density, position/power, centrality, clique, and cohesion measures, as well as sociocognitive
dynamic patterns of discourse, according to relationships between students, students/facilitator,
and knowledge work with facts and/or emergent ideas. A comprehensive Knowledge Building
study involving traditional measures of success and social network structural and content
analyses has not been conducted in continuing medical education.
Evidence of success of educational innovations is often provided through proof of knowledge
gains and positive feedback on program satisfaction surveys. These are important learning
outcomes but are rooted in an individual metaphor. Despite the importance of these measures,
they provide insufficient descriptions to help us understand “how” and “why” innovations
succeeded (Kanter, 2008), and offer no window into necessary representations of socially
mediated interactivity. What is required are elaborated, detailed descriptions of the nuances,
relationships, and interrelationships and the sociocognitive dynamics of Knowledge Building—
to provide better understanding of community engagement and structures that scaffold collective
Knowledge Building.
1.6 Significance
This study is significant because it will explore and clarify Knowledge Building and the
relationship between traditional measures of educational success and social network structural
and content assessments, and how they contribute to the advancement of current
conceptualizations of continuing medical education. Continuing medical education has often
been characterized as an individual endeavour, conceived within the constraints and limitations
of self-assessment, self-directed learning, and self-reflection and didacticism (Davis & Davis,
2010; Davis et al., 2006; Davis et al., 1999; Silver, Campbell, Marlow, & Sargeant, 2008), as
opposed to socially mediated assessment. Many continuing medical education courses are large-
group lectures and/or conference presentations (Davis et al., 1999; Davis, Davis, & Bloch, 2008).
Numerous web-based courses are available for individual online learning (Cook et al., 2008;
Curran, Lockyer, Sargeant, & Fleet, 2006; Fordis et al., 2006). Few provide opportunities for
collective discourse; and even fewer, apart from what I describe in this study, have been
designed specifically to provide opportunities for collective Knowledge Building in continuing
medical education.
Knowledge Building in Continuing Medical Education … 12
This will be the first known study to describe social network structural dimensions and
sociocognitive dynamics of sustained, collective Knowledge Building in continuing medical
education; it will describe a social network of physicians working with knowledge and ideas in
an online problem space, over and above accomplishing predetermined learning objectives; and,
as well, corresponding assessments of traditional outcome measures and social network
structural and content analyses of Knowledge Building. This study aims to describe socially
mediated relationships and provide evidence of the potential of Knowledge Building in
continuing medical education.
1.7 Research Questions
This study will be guided by the following research question and a set of subquestions. The
research question is:
Does Knowledge Building improve physicians’ knowledge and understanding of
palliative care in a Web-based continuing medical education course, and if so, what
social network structural relationships and sociocognitive dynamics support knowledge
improvement, and contribute to democratization and metadesign of Knowledge Building
in continuing medical education?
Subquestions organized in the following three clusters will frame this study:
Cluster 1 Traditional outcome measures
Cluster 2 Performance over and above traditional measures (beyond learning as
traditionally conceived and measured)
Cluster 3 Sociocognitive dynamics that enable work over and above traditional
learning
1.7.1 Research Clusters and Subquestions
Cluster 1: Traditional outcome measures.
1. Did students’ pain knowledge improve from pre to posttest and if so, is this
improvement significant and what is the effect size?
Knowledge Building in Continuing Medical Education … 13
2. Traditionally continuing medical education is a self-directed or didactic experience;
what are participants’ attitudes and opinions toward collaborative online knowledge
building in the End-of-Life Care Distance Education Program?
Cluster 2: Performance over and above traditional measures (i.e., beyond learning as
traditionally conceived and measured).
3. What are participants’ online activity and interactivity measures, by module, by
group, and by year, in the 2004–2005 pilot and 2005–2009 study, using the
Knowledge Forum analytic toolkit (ATK)?
4. What participation patterns emerge in the 2008/2009 Group 1 and Group 2 (selected
for analysis because of largest pre-/posttest knowledge gains), through use of the
graphical ATK contribution build-on/read assessment tool and corresponding social
network analyses of network density? Is there a significant difference in 2008/2009
Group 1 and Group 2 pain knowledge scores from pre- to posttest and is there a
significant difference between groups? If so, what is the relationship between these
differences and patterns of social network density?
5. What are the social network structural differences between groups that support
increased knowledge improvement and how are these differences related to
centralization, cohesion, and/or democratization of participation and ideas? Is there a
relationship between social network dimensions?
Cluster 3: Questions regarding the sociocognitive dynamics that enable work over and above
traditional learning.
6. What are the social network relationships between structural position, power—
defined as centrality of ideas—and knowledge improvement; how are these
relationships reflected in facilitator/student sociocognitive dynamics, through
emergent themes within the discourse (beyond learning objectives), in the complexity
of discourse, and by indicators of Knowledge Building?
6.1 What is the relationship between facilitator and students position/power? Whose
ideas are at the core, middle, and periphery of the space?
6.2 Can the facilitator’s stance be described as a monitor, mentor, partner or expert
in the sociocognitive knowledge work? Knowledge work is advanced through
Knowledge Building in Continuing Medical Education … 14
statements and questions. Who asks the questions that drive knowledge work—
students or the facilitator?
6.3 Is there evidence of metadesign knowledge work, as expressed through
emergent themes/ideas, beyond the predefined learning objectives? What
themes emerge?
6.4 Is discourse clinically complex as defined by elaboration or compilation of
knowledge and ideas?
6.5 Is there evidence, in the discourse, of Knowledge Building indicators such as
community responsibility for knowledge advancement, democratization of
knowledge, epistemic agency, improvable ideas, and work with real ideas,
authentic problems? And are indicators of Knowledge Building metadesign
present, as evidenced by intentionality and strength of epistemic agency, work
with community knowledge, and emergent ideas?
The goal of this research study is to demonstrate improvements according both traditional
learning and nontraditional Knowledge Building measures and to provide a meaningful
description of Knowledge Building, in a socially mediated discourse space, and the potential for
change in traditional power structures, in favour of democratization of ideas, cocreation of
knowledge, and metadesign of continuing medical education.
1.8 Dissertation Organization
This study follows traditional research reporting structures of introduction, literature review,
methods, results, discussion, and conclusions. Chapter 1 sets out the background, purpose, and
significance of the study and provides the overarching structure, based on the three clusters of
research subquestions, as delineated above. Chapter 2 is a review of the literature and is also
divided into three sections corresponding to the study clusters. The first section provides an
overview of current issues in continuing medical education, the second reviews key concepts of
Knowledge Building and studies employing social network analyses, and the third section
summarizes the literature on social network analyses relevant to the context herein. Chapter 3
describes the study methods; it begins with an overview of the case-based, (media pedagogic
design of the End-of-Life Care Distance Education program; this section is followed by details of
the research methods. Research methods are divided into three sections, based on the conceptual
Knowledge Building in Continuing Medical Education … 15
categories set out in the clusters of research subquestions. Chapter 4 presents results of the pilot
study. Chapters 5, 6, and 7 provide results of the research subquestions based on the three
clusters. Thesis results are discussed in Chapter 8, and conclusions in Chapter 9 (see Table 1 for
a summary of all the above). Validation of pre-/posttest questionnaires can be found in the
Appendices.
Knowledge Building in Continuing Medical Education … 16
Table 1
Research Clusters and Organization of Dissertation
Introduction and
Research
Clusters
Literature
Review
Methods
Results
Discussion
and
Conclusions
CLUSTER CHAPTER 1 CHAPTER 2 CHAPTER 3 CHAPTER 4 (pilot study) CHAPTERS 8/9
CHAPTER 5
1 Traditional
measures
Continuing
medical
education
Pre-/posttests, attitudes and opinions survey
across all years, 2005–2009
Results of knowledge gains, attitudes, and
opinions, 2005–2009
Synthesis and
significance
CHAPTER 6
2 Performance
over and above
traditional
measures
Knowledge
Building
and social
network
studies
Activity, interactivity measures across 5
years and in-depth SNA using KF/ATK
measures of two groups in final year,
2008/2009; SNA of centrality, cliques,
cohesion using Netminer 3 software
Results of read/write activity, build-on
interactivity, across years and comparative
results of social network density of notes
built-on and read in final year, 2008/2009;
comparative results of centrality measures
with and without the facilitator across all
2008/2009 modules
Synthesis and
significance
CHAPTER 7
3 Sociocognitive
dynamics that
enable work
over and above
traditional
learning
Knowledge
Building
and social
network
analysis
(SNA)
SNA of position centrality/power of ideas
and within-note analyses
⋅ Facilitator stance and examination of
who asks questions that drive the
discourse? (Demonstrates intentional
KB/epistemic agency and collective
responsibility)
⋅ Themes identification beyond learning
objectives
⋅ Complexity of clinical discourse
⋅ Modules to be scored for evidence of
Knowledge Building indicators
Results of position centrality/power of
ideas in relationship to knowledge gains
- Facilitator/student structures, facilitator
stance as partner and expert
- Emergent ideas and themes are identified
beyond learning objectives
- Discourse is categorized as elaborated
and compiled.
- Principles/indicators of Knowledge
Building are scored/evident; exemplars are
provided
Synthesis and
significance
Knowledge Building in Continuing Medical Education … 17
CHAPTER 2
LITERATURE REVIEW
2.1 Introduction
Recent research and reports on the ineffectiveness of continuing medical education and calls for
reform, new ideas, new applications, and alternative strategies for improvement were described
in the first chapter and will be elaborated upon in this chapter, the literature review.
Continuing medical education courses, for physicians in practice, are typically designed for the
transmission of facts, new treatments, recent discoveries, and understanding of best-practice
guidelines, aimed at improving physician practice and patient care. There is no doubt that the
transmission of the most-up-to date knowledge is essential. However, this thesis argues that the
knowledge transmission approach is limiting and suggests that in addition to learning the latest
facts and knowledge of best practice guidelines, physicians should be provided with
opportunities to go beyond best practices (Scardamalia, 2002), through sociocognitive
Knowledge Building.
The goal of this thesis is to show improvements according to both traditional and nontraditional
measures. As noted previously, this thesis will describe (a) traditional learning outcomes based
on individual measures, (b) performance over and above traditional measures (i.e., beyond
learning as traditionally conceived and measured), and (c) sociocognitive dynamics that enable
work over and above traditional learning. The literature review has been structured around these
three research clusters, to inform an understanding of each area and identify knowledge and
research gaps. The first section will provide necessary definitions and background on continuing
medical education. The second section will provide an overview of Knowledge Building theory,
and the third section will review social network analysis and other strategies of network
analyses. The literature review will conclude with a summary of implications for Knowledge
Building in continuing medical education.
Knowledge Building in Continuing Medical Education … 18
2.2 Cluster 1: Traditional Conceptualizations of Continuing Medical Education
A comprehensive history of the evolution of continuing medical education in Canada has
recently been published by Dr. W. Dale Dauphinee (2011) tracing the origins of concepts and the
evolution of structures. Chapters from the same book (Wentz, 2011) provide details too
numerous for inclusion here, on such subjects as accreditation of continuing medical education
(Woollard, 2011), standards for specialists defined by the Royal College of Physicians and
Surgeons of Canada (Campbell & Parboosingh, 2011); and standards, and for family physicians
as defined by the College of Family Physicians of Canada (Dunikowski, 2011).
The first section of the literature review will provide an overview of traditional
conceptualizations of continuing medical education and current issues, important to this research
study.
2.2.1 Continuing Medical Education for Individual Lifelong Learning
2.2.1.1 Definition. The Canadian Institutes of Health Research (2003) have defined continuing
medical education as “any and all ways by which physicians learn and maintain their
competence” and specifically refers to “education after certification and licensure” (Davis,
Evans, et al., 2003). This definition of continuing medical education is often extended, drawing
relationships to professionalism and practice improvement, through two other commonly used
terms, continuing professional development and knowledge translation (Davis, Barnes, & Fox,
2003; Davis, Evans, et al., 2003; Graham et al., 2006; Wentz, 2011). Continuing professional
development “embodies both professional learning and personal growth” (Davis, Evans, et al.,
2003). Some definitions distinguish continuing medical education from professional
development by including concepts of performance improvement through learning in clinical
practice settings (Davis, Barnes, et al., 2003; Bennett et al., 2000). Often, notions of continuing
medical education and continuing professional development are linked together, as in the
recently revised American Medical Association (2010) definition, which is as follows:
CME consists of educational activities which serve to maintain, develop, or increase the
knowledge, skills, and professional performance and relationships that a physician uses to
provide services for patients, the public or the profession. The content of CME is the
body of knowledge and skills generally recognized and accepted by the profession as
within the basic medical sciences, the discipline of clinical medicine and the provision of
health care to the public. (p. 2)
Knowledge Building in Continuing Medical Education … 19
Evaluation of knowledge translation is beyond the scope of this study. Knowledge translation is
defined and distinguished as “the exchange, synthesis and ethically sound application of
knowledge . . . to accelerate the capture of benefits of research . . . through improved health,
more effectives services and products and a strengthened health care system” (Davis, Evans, et
al., 2003). The definition of knowledge translation links continuing medical education to health
care outcomes (Davis, 2011; Dorman & Miller, 2011).
The College of Family Physicians of Canada (n.d.) has stated: “As professionals, family
physicians are obliged to remain current on advances and trends in medicine and health care
delivery. This is achieved through participation in a variety of activities that constitute
continuing professional development (CPD)” (para. 1). The End-of-Life Care Distance
Education Program received full accreditation from The College of Family Physicians of Canada
and was further reviewed and accepted by University of Toronto’s Office of CEPD at the Faculty
of Medicine, which uses the following definition: “Continuing education is a process of lifelong
learning for health professionals to expand and fulfill their potential, retain their capacity to
practice safely and effectively, and to meet the healthcare priorities of the population and broader
health systems” (University of Toronto, 2011, p. 6).
The Office of CEPD recently published a strategic plan for 2011–2016 called “Better Education,
Better Outcomes” (University of Toronto, 2011). The plan calls for a change that includes
moving from self-learning and didactic formats to collaborative learning and social network
opportunities, so as to better integrate and sustain continuing medical education for lifelong
learning. This shift is intended to move from episodic learning, i.e. discrete educational events
such as lectures and workshops at conferences, to longitudinal, continuous interactions, such as
those involving online social network communities and practice-based settings.
Strategic directions identified by the 2011–2016 plan are:
1. Enhance best practice and faculty development for continuing education
2. Advance research, innovation, and scholarship
3. Strengthen practice-based education
4. Foster quality improvement in continuing education
5. Promote patient and public engagement (University of Toronto, 2011, pp. 7–9)
Enabling strategies include entrepreneurship, information technology and social media,
communities of practice, partnerships, and sustainable funding (University of Toronto, 2011, pp.
Knowledge Building in Continuing Medical Education … 20
10–11). It is clear that technology and concepts of communities to advance knowledge figures
prominently in this new vision.
Approximately 27,000 individuals participated in various events/programs in 2009–2010 offered
by the Office of CEPD; however, as the aggregate data over five years demonstrates, most
participated in live events and very few participated in web-based learning (University of
Toronto, 2011, p. 4, fig. 2). Of the online courses offered most are designed for individual
learning; few are designed for online collaborative discourse.
The new vision embodied by the Faculty of Medicine’s strategic plan is well aligned with the
concept of knowledge-creation concepts of Knowledge Building: “The Office of CEPD is
seeking to change problem solving into possibilities for breakthrough thinking. This included
thinking about continuing education as a type of creative experiment” (University of Toronto,
2011, p. 1).
2.2.1.2 From individual to socially mediated participatory metadesign. Over the last decade
or more, e-learning continuing medical education courses have become popular. Web portals
such http://www.MDcme.ca offer numerous online courses in various clinical content areas. A
review of this site indicates that most courses are individually accessed online and aimed at
learning (Knowles, 1975); few use discussion board environments or are aimed at computer
supported collaborative learning (Brown & Duguid, 2000); none were found that focus on
collaborative Knowledge Building (Scardamalia & Bereiter, 2003a).
Various e-learning and information technology tools for individual physician use have been
developed by Canadian regulatory bodies. In 2000, the Royal College of Physicians and
Surgeons of Canada upgraded their Maintenance of Competence (MOCOMP) program and
established a mandatory Maintenance of Certification (MOC) program. Physicians were required
to document their continuing professional development activities and to facilitate this, the
College developed a Web Diary and a Question Library (Campbell & Parboosingh, 2011). A few
years later these two programs were amalgamated into one site called Mainport for individual
physicians’ record keeping and MOC credits.
Similarly, the College of Family Physicians of Canada has sponsored various innovative
programs using information technology, such as the eCME Resource Centre (Dunikowski,
Knowledge Building in Continuing Medical Education … 21
2011). This resource site makes available dozens of online course. Two well-established web-
based programs sponsored by the College of Family Physicians are the Self Learning™ suite and
the Pearls™ (n.d.) program. Self Learning™ is web published in issues, each of which contains a
series of multiple-choice questions and short-answer management problems; a concise summary
highlights major points, with links to PubMed abstracts and full text articles. Self Learning™
had more than 6,000 subscribers worldwide in 2009 (Dunikowski, 2011). Pearls was also
introduced in a web-based version in 2009. It is “a self-directed, structured, learning activity”
(Dunikowski, 2011, p. 246) that can be completed by physicians as an evidence-based practice
reflection exercise:
Participants use the medical literature to answer their own practice questions and later
reflect on how well decisions have been integrated into their practice. Based on Donald
Schon’s reflective learning cycle, a Pearl’s exercise has five steps: formulating a practice-
based question, seeking information from peer-reviewed literature, evaluating and
critically appraising the articles, making a practice decision based on what was learned,
and indicating the effect of this decision on patient outcome. (Dunikowski, 2011, p. 246)
Physicians are awarded Mainpro-C credits for Pearls exercises. Concerns about self-directed
learning and self-evaluation in these types of continuing medical education scenarios have been
published (Regehr & Eva, 2006) and prompted the notion of “directed self-assessment” by the
Committee on Accreditation of Continuing Medical Education (ACCME, n.d.; Woollard, 2011),
and more recent recommendations for collective knowledge work and community engagement,
as in the Office of CEPD strategic plan (University of Toronto, 2011), discussed above.
Ill-structured knowledge domains are conceptually complex and irregular across cases (Spiro,
Feltovich, Jacobson, & Coulson, 1991). It is well known that medicine has been characterized as
an ill-structured domain (Spiro, Coulson, Feltovich & Anderson, 1998; Spiro et al., 1991).
Interactions made possible by computers has been identified as the ideal medium for criss-
crossing ill-structured domains (Spiro & Jehng, 1991). Although Spiro and Jehng (1991) were
not referring to Knowledge Building in Knowledge Forum, the notion of benefits from
interactivity and linking of ideas in an ill-structured domain is clear. Connecting ideas is most
productive when socially constructed and aimed at improvement of ideas (Scardamalia &
Bereiter, 2003a, 2003b).
As we have seen from the literature reviewed above, physicians usually do not have the
opportunity to collectively participate in Knowledge Building—to create and design their own
Knowledge Building in Continuing Medical Education … 22
collective continuing education experiences. Physicians’ contributions to the metadesign of their
continuing medical education experiences may allow these experiences to become more relevant
and emergent, to better meet their needs, scaffold continuous improvability, and elevate
community knowledge. Various studies in other contexts have indicated that metadesign
translates into improved practice and care—addressing the knowledge to practice gap (Fischer,
2009a, 2009b, 2010; Fischer & Giaccardi, 2006; Fischer & Konomi, 2007).
The notion of improving physician participation in continuing education is not new; what is new
is the idea of elevating physician participation in continuing education as a metadesigner of the
curriculum and cocreator of knowledge, with the intended purpose of improving ideas of value to
the community. This may be possible in an environment that supports collective knowledge
building, where continuing medical education can assume an open problem space, a more
emergent structure, and more symmetrical opportunities to engage in sociocognitive work in
belief and design modes, as elaborated below. It is posited that through theory-based designs
within a knowledge building perspective, continuing medical education may become more
authentic and democratized, promoting changes in performance and idea innovation.
In the next section I will review selected literature on Scardamalia and Bereiter’s theory and
concepts of Knowledge Building. It is important to note that there is a much broader literature on
knowledge building that is not reviewed, as it is beyond the scope of this paper. To put this
broader literature in perspective, a web search of the phrase “knowledge building” a decade ago
showed less than 10,000 entries. Within five to years it had reached the same level as
“knowledge creation”—its more popular synonym in the business world: about 500,000 entries.
It has continued to grow as an area of intense research, keeping pace with its counterpart,
knowledge creation, and now shows well over a million links.
There seems to be growing awareness that knowledge creation must extend beyond the realm of
intellectual elites—that citizens worldwide need to have a part in a knowledge-creating culture.
Giving them a role requires something that expands the idea of knowledge creation and the
notion of individual genius often associated with it. That challenge is being tackled by a
worldwide community, as suggested at the Computer-Supported Collaborative Learning
Conference held in 2011 in Hong Kong (CSCL2011, 2011). Knowledge building was one of five
major strands, with over 50 research reports presented. Knowledge Building was earlier
Knowledge Building in Continuing Medical Education … 23
represented in the Cambridge Handbook of the Learning Sciences (Sawyer, 2006) as one of five
foundational approaches within the learning sciences. The strong collaborative framework for
this work is reflected in special issues of the Canadian Journal of Learning and Technology
(Scardamalia & Bereiter, 2010) devoted to knowledge building and QWERTY: Interdisciplinary
Journal of Technology and Culture, with approximately 12 disciplines represented, as well as
multiple cultural contexts and research involving students from elementary to tertiary education,
teacher education, and teacher-researcher-government partnerships. Topics range over
knowledge building theory, models, and theory-based principles; tools and methodologies;
knowledge-building dialogue; applications (e.g., Aboriginal education and totally online and
blended learning); and results and outcomes (including a quantitative model for the analysis and
the assessment of the collaborative structure of a knowledge-building community and assessment
of community knowledge). There is an Italian Association for Knowledge Building, a Centre for
Research on Knowledge Building and Technology at the University of Helsinki, and major
knowledge-building research programs spanning the Americas, Europe, and Asia.
2.3 Cluster 2: Performance Over and Above Traditional Individual
Outcomes, Through Collective Knowledge Building
This research cluster has its basis in the literature of collective Knowledge Building, as defined
by Scardamalia and Bereiter (2002, 2003a, 2006). Knowledge Building theory, pedagogy, and
technology demonstrate the theme of this second cluster: performance over and above traditional
outcomes (beyond learning as traditionally conceived and measured), through collective
Knowledge Building. This section begins with a review of selected Knowledge Building studies
in medical and health sciences education that demonstrate the challenges of research in this
context.
2.3.1 Knowledge Building Research in Medical and Health Sciences Education
Various Knowledge Building research studies have been conducted in medical and health
sciences education; however, a paucity of research exists particularly in the area of continuing
medical education.
Two recent doctoral research studies conducted by Punja (2007) and Mylopolous (2007)
examined Knowledge Building in the context of medical education. Punja’s study involved four
Knowledge Building in Continuing Medical Education … 24
case studies across three different levels of medical education: undergraduate medical education,
graduate residency, and postlicensure continuing medical education. Punja conducted a brief
pilot study in a psychiatric continuing medical education course. Knowledge Forum was used as
an add-on to support additional discussion, beyond face-to-face sessions; the discourse never
built momentum. Punja found similar results across all settings, characterizing each as teacher
dominated, and concluding that teacher-generated activities made it difficult for students to
assume epistemic agency for their knowledge work. Barriers to Knowledge Building included
the belief that knowledge innovation should occur after acquisition of foundational knowledge,
as opposed to operating in parallel and providing a better basis for acquisition of foundational
knowledge.
Similar barriers were identified in Mylopoulos’s (2007) grounded theory dissertation study and a
related publication (Mylopolous & Scardamalia, 2008). Mylopolous conducted semistructured
interviews with a sample of 15 clinical faculty members at University of Toronto’s Faculty of
Medicine. She demonstrated that clinical faculty held the view that innovation is developed
through research and then diffused through the community for adoption. Hence, in this view,
innovation is removed from personal responsibility and daily practice. Mylopolous and
Scardamalia concluded that these perceptions of innovation typically limited participants’
engagement in collaborative processes central to knowledge building.
Studies such as these inform our understanding of how to better design the social infrastructure
critical to support collective community online work (Bielaczyc, 2006). Bielaczyc (2006)
described four dimensions: “(a) cultural beliefs, (b) practices, (c) socio-techno-spatial relations,
and (d) interaction with the “outside world” (p. 301). Both Punja’s (2007) and Mylopoulos’s
(2007) studies point to the substantial challenges that need to be addressed to change perceptions
and engagement with knowledge innovation.
Some other recent studies in the health sciences explored the potential of Knowledge Building:
in a pharmacy course, typically taught through lectures (Sibbald, 2009): in a hospital-based
interprofessional health sciences teams (Russell, 2002, 2005); and in an undergraduate
interprofessional health sciences education pain curriculum (Lax, Scardamalia, Watt-Watson,
Hunter, & Bereiter, 2010; Lax, Watt-Watson, Pennefather, Hunter, & Scardamalia, 2002, 2003a,
2003b).
Knowledge Building in Continuing Medical Education … 25
Each of these studies uncovered substantial challenges and the need for research demonstrating
that there need not be tradeoffs between learning and knowledge building—but rather these
forces may be mutually reinforcing. And none of these studies examined social network
relationships and sociocognitive dimensions of sustained knowledge building in continuing
medical education. The current study addresses this gap in the literature, to illuminate the
community relationships that support knowledge improvement beyond individual knowledge
growth. What follows is a detailed overview of the theory, pedagogy, and technology of
Knowledge Building.
2.3.2 Distinction Between Learning and Knowledge Building
Central to education becoming a knowledge-creating process is better integration of belief mode
(argumentation, persuasion, and critical analysis) and design mode (theorizing, invention, and
idea improvement; Bereiter & Scardamalia, 2003b). Currently, work in design mode is seriously
underrepresented in continuing medical education. Work in design mode is fundamental to
knowledge building. Designers do not contemplate the possibility of a final state of perfection
(Petroski, 2003); neither do knowledge building students and teachers. As Scardamalia and
Bereiter (2003a) have noted, knowledge building is associated with innovation, intellectual
capital, and intellectual property, and is synonymous with knowledge creation. Its growth
alongside knowledge creation reflects a need for understanding processes by which all citizens
can engage in knowledge creation and educational institutions operate as knowledge-creating
organizations.
It is posited that the didactic paradigm of most current continuing medical education needs to
allow for more authentic, relevant, design-mode work. Continuing medical education reframed
as a trajectory of life-long knowledge building may add new understanding to help effect change
and inform us about the sociocognitive dimensions and supports required for the collective
advancement of knowledge. Knowledge Building offers a framework for pedagogic design and
evaluation (Scardamalia & Bereiter, 2006). Knowledge Forum is an Internet-based communally
accessible learning technology developed by Scardamalia and Bereiter to support collaborative
Knowledge Building. Knowledge Building provides a theoretical and pedagogical framework
and Knowledge Forum provides an easily accessible collaborative communication technology to
support educational design and evaluation. This combination of theory, pedagogy, and
Knowledge Building in Continuing Medical Education … 26
technology for design and evaluation allows us to address continuing medical education in
dynamic new ways.
Scardamalia and Bereiter (2003a) have defined Knowledge Building as follows:
Knowledge Building may be defined as the production and continual improvement of
ideas of value to a community, through means that increase the likelihood that what the
community accomplishes will be greater than the sum of the individual contributions and
part of broader cultural efforts. (p. 1370)
Thus, Knowledge Building is seen as pervasive, not limited to education, but applicable to
society on the whole and practice-based communities, such as those addressed in the study
herein.
Scardamalia and Bereiter (2003a) have clearly distinguished learning (as private knowledge)
from Knowledge Building (as public knowledge):
Learning is an internal, unobservable process that results in changes of belief, attitude or
skill. Knowledge building, by contrast, results in the creation or modification of public
knowledge—knowledge that lives ‘in the world’ and is available to be worked on and
used by other people. Of course creating public knowledge results in personal learning,
but so does practically all human activity. (p. 1371)
Knowledge Building is further distinguished from learning by disclaiming the “mind as
container” metaphor and “jug to mug” analogies for teaching and learning (Bereiter &
Scardamalia, 1996, p. 486). In Bereiter and Scardamalia’s (1996) paper, “Rethinking Learning,”
the authors used “the term ‘knowledge building’ to refer to the production of knowledge
objects—objects that in Popper’s scheme occupy World 3” (p. 494). They indicated that
Knowledge Building is activity directed toward changes in World 3 (through ideas, theories, and
interpretation), while learning is directed toward World 2 (states of knowing). With this comes
the notion of “moving ideas to the centre”—a pivotal and vivid conception describing how to
work with knowledge objects and artifacts (Scardamalia, 1999). Using these distinctions,
Bereiter and Scardamalia (1996) have indicated that learning can be seen as complementary to
Knowledge Building. In their article, “Beyond Bloom’s Taxonomy,” Bereiter and Scardamalia
(1999) set out a series of levels of objectification for knowledge work that go beyond Bloom’s
(Bloom, Engelhart, Furst, Hill, & Krathwohl, 1956) original or revised (Anderson & Krathwohl,
2001) educational objectives.
Knowledge Building in Continuing Medical Education … 27
A complete history of Scardamalia and Bereiter’s ideas has been published (Scardamalia &
Bereiter, 2010). Their definition and development of Knowledge Building theory resonate from
years of work in the domains of cognitive psychology and education, guided by philosophical
underpinnings, influenced by new technologies, the Internet, and concerned with rigorous
research and design methods to support innovation in the knowledge age. They and an ever-
expanding international knowledge-building community of teachers, scholars, and researchers
have advanced this work substantially.
2.3.3 Historical and Cultural Relevance of Knowledge Building
In 1989, Scardamalia and Bereiter and their colleagues McLean, Swallow, and Woodruff at the
Centre for Applied Cognitive Science, at the Ontario Institute for Studies in Education (now part
of the University of Toronto), published a seminal paper, “Computer-Supported Intentional
Learning Environments” (CSILE). The authors described principles based on cognitive science
research for the design of computer environments. These computer environments were intended
to scaffold students’ intentional control over their own learning; that is, the setting of cognitive
goals, self-monitoring, strategies for comprehension and organization of knowledge.
Intentional learning and CSILE were described in parallel. CSILE was initially developed as a
hypermedia system using text and graphic notes for groups of students to share their thoughts.
Scardamalia et al. (1989) called this process “building a collective database (knowledge-base)
. . . CSILE stores the thoughts entered by each student, making them available to everyone” (p.
52). Results of early trials using CSILE with university master’s level students provided
encouraging feedback on processing and reprocessing of thoughts on research literature. CSILE
was implemented as a networked computer system, in numerous classrooms at all grade levels.
Scardamalia et al. (1989) provided a review of cognitive research of the day and noted, along
with many researchers, the need for active rather than passive learning strategies. But while
many researchers were calling for intelligent tutoring systems, Scardamalia et al. declared:
It is not the computer that should be doing the diagnosing, the goal-setting, and the
planning, it is the student. The computer environment . . . should be providing the
facilitating structure and tools that enable students to make maximum use of their own
intelligence and knowledge. (p. 54)
Knowledge Building in Continuing Medical Education … 28
This concept of procedural facilitation underpins Scardamalia and Bereiter’s previous work on
written composition, Palincsar and Brown’s (1984) “Reciprocal Teaching,” Brown and
Palincsar’s (1989) “Guided, Cooperative Learning” approach, and Collins, Brown and
Newman’s (1989) “Cognitive Apprenticeship.” Concepts of procedural facilitation have been
expanded and exist in current forms as scaffolds in Knowledge Forum and strategies for
collective Knowledge Building. A series of design principles for CSILE grew out of these initial
experiments. They include:
· Make knowledge-construction activities overt
· Maintain attention to cognitive goals (rather than task goals)
· Identify knowledge “lacks” and provide positive consequences
· Provide process-relevant feedback (as opposed to product feedback)
· Encourage learning strategies as opposed to rehearsal/memorization
· strategies (i.e. drill and practice approaches)
· Encourage recursive, multiple-pass strategies through information
· Support varied ways of knowledge organization and representation
· Encourage unlimited use and examination of existing knowledge
· Provide opportunities for reflectivity and individual learning styles
· Facilitate transfer of knowledge across disciplinary contexts and
· curriculum boundaries
· Give students more responsibility for contributing to each other’s learning
(Scardamalia et al., 1989)
Scardamalia et al. (1989) indicated that identified knowledge “lacks” should be met with positive
consequences, possibly from program feedback, resulting in enhanced possibilities for
overcoming these lacks and fully achieving the learning goals. Current notions of embedded and
transformative assessment (Scardamalia, 2002) can be traced back to these early roots. Many of
the concepts articulated are pertinent to current educational contexts and the context of the study
herein.
In Bereiter and Scardamalia’s (1990) paper, “An Architecture for Collaborative Knowledge
Building,” the authors described the pivotal concept of community, and the importance of a
Knowledge Building community as opposed to individual knowledge work. They indicated that
the guiding idea for design and application of CSILE was to support Knowledge Building
communities, which they likened to successful scientific research groups. Influential educational
concepts of constructivism, “the idea that knowledge is a human construction, not something that
exits to be revealed and transmitted”; sociocultural activity, “as the medium through which
knowledge construction takes place”; and apprenticeship, “in that much of the skill of the young
Knowledge Building in Continuing Medical Education … 29
scientist is acquired by working along with a more mature scientist” were described (Bereiter &
Scardamalia, 1990, p. 42).
Bereiter and Scardamalia’s (1990) continued evolution of design principles can be summarized
as follows:
1. Objectification. Reference is made to Carl Popper’s concept of the objectification of
knowledge. Knowledge should be treated as an object outside the mind to be criticized,
modified, related and compared from different viewpoints in different contexts. Scardamalia and
Bereiter (1990) stated:
In this way the danger of knowledge being treated as an absolute, devoid of human bias,
can be minimized. Community critique, examination and belief revision are central to
knowledge work. The problem in conventional classrooms is that knowledge is generally
transmitted as an absolute and critique is subordinated to tasks and activities. (p. 44)
2. Progress. Knowledge Building should evoke what Whitehead called “disciplined progress”
that is evident to participants. Much of learning in conventions situations is cyclical rather than
progressive.
3. Synthesis. CSILE should support continual synthesis and progressive construction of higher
order knowledge representations and understandings. This process should be formative rather
than summative. CSILE should counteract the tendency of hypermedia to create a loosely
connected network of ideas lacking organization, synthesis and progress.
4. Consequence. Positive reinforcement as a result of knowledge processing operations is
advocated. In a scientific community reinforcements include citation, comment on, use and
confirmation of one’s ideas by another. Such reinforcements could be provided in CSILE. Ideas
advancement can be perceived by the entire community in CSILE.
5. Contribution. Distributed discourse is intended to enhance depth of understanding through
multiple perspectives and to raise the knowledge of the collective, not just the individual.
Development of expertise is defined by a progressive problem solving process; problems are
continually reformulated at increasing levels of complexity (Bereiter & Scardamalia, 1990, p.
45).
Knowledge Building in Continuing Medical Education … 30
These concepts, particularly communal development of expertise, stand in contrast to the
development of individual expertise defined in terms of a staged, five-step, novice-to-expert
model (Dreyfus & Dreyfus, 1986). For Bereiter and Scardamalia (1993), the problem with such a
staged model is that expertise is defined as an end state. Instead, knowledge building reframes
development of expertise using a process model of expertise, characterized by reinvestment of
mental resources at higher and higher levels in a collaborative and progressive problem-solving
process (Bereiter & Scardamalia, 1993a; Scardamalia, 2002).
Scardamalia and Bereiter’s notion of intentionality remains key to concepts of continual
improvability. Intentionality has evolved into the robust principle of Knowledge Building known
as epistemic agency (Scardamalia, 2002). Examination of the early definitions of intentional
learning are informative and still relevant. Scardamalia and Bereiter (1989) indicated:
We use the term intentional learning to refer to cognitive process that have learning as a
goal rather than an incidental outcome. . . . Whether intentional learning occurs is likely
to depend on both situational and intrinsic factors - of what the situation afford in goal
attainment opportunities and on what the student’s mental resources are for attaining
those goals. Thus, focusing on intentional learning provides a natural way of coordinating
the two relevant research traditions . . . learning situations and . . . learning skills. (p. 363)
Bereiter and Scardamalia (1989) distinguished between learning through problem solving and
learning as problem solving. Learning thorough problem solving results from cognitive
operations applied to knowledge states in an effort to advance knowledge toward the goal. In
contrast, learning as problem solving implies that the goal itself is a learning goal. Learning
through problem solving is aimed at acquiring a deeper understanding and competence for
purposes of lifelong learning (Bereiter & Scardamalia, 1989, pp. 365–366). Learning as problem
solving is aimed at completing learning tasks and activities:
Metaknowledge, or knowledge about knowledge and personal assessment and
understanding of one’s own knowledge and skills is needed for intentional learning. The
three major metaknowledge needs are characterized as: 1. A problem solving framework
for approaching learning, that is an “executive structure”; 2. Awareness of the functional
potential of knowledge, that is that knowledge can be “transformative.” Rather than
considering knowledge as a new fact to be added to a collection, it can set off a reflective
process to change a central belief, to connect facts to build new understanding; 3.
Strategies for identifying deficits in one’s own knowledge is a vital part of intentional
learning. Without it, the only kind of learning goal one can set is to learn more about a
topic. ...When learning is approached within a problem solving framework, identifying
what one does not know becomes a variety of problem-finding. (Bereiter & Scardamalia,
1989, pp. 375–377)
Knowledge Building in Continuing Medical Education … 31
Identification of problems is considered to be “an active, strategically guided process,” not a
simple whim of curiosity.
The conclusion that Bereiter and Scardamalia (1989) arrive at is:
In order to learn what is ostensibly being taught in school, students need to direct mental
effort to goals over and above those implicit in the school activities. Without such
intentional learning, education degenerates into the doing of schoolwork and other
activities. (p. 385)
This cautionary statement applied to education of youth can also be applied to that of adults and
the context of continuing medical education.
The pursuit of cognitive goals, over and above the requirements of tasks is what Bereiter and
Scardamalia have called intentional learning. Strategies to empower students to pursue cognitive
goals can be met by teachers teaching students about metaknowledge; teachers progressively
turning over high level functions (e.g., reframing questions or summarizing) to students; teachers
modelling the process of setting level, more meaningful cognitive goals beyond schoolwork.
Higher-level cognitive goals require self-assessment of level of constructive effort since it cannot
be judged as a task.
Scardamalia and Bereiter’s (1989) framing of all levels of education as a Knowledge Building
community shifts responsibility, in the sense that there exists not only a body of knowledge to be
acquired, but a dedication towards contribution and advancement to the body of knowledge, over
and beyond individual achievement. This framing is fitting for current examination and
reconceptualization of continuing medical education, and addresses potential physician
commitment to not only to lifelong learning, but to pervasive Knowledge Building.
2.3.4 Belief Mode and Design Mode
“We do not suggest that ‘design mode’ should replace ‘belief mode’ in schools. Both are
important. . . . Knowledge creation depends on moving back and forth between belief and design
questions in ways that maintain progress in idea improvement” (Bereiter & Scardamalia, 2007,
pp. 25–26). These concepts are complex and demand further explanation.
Knowledge Building has been characterized as sociocognitive “work at the edge of knowledge”
(Scardamalia, 2003b). In order for work to occur at the current edge of knowledge, students and
Knowledge Building in Continuing Medical Education … 32
communities must have an up-to-date and deep of understanding of the belief-mode edge of
ideas and artifacts. Students must know what the facts and larger concepts are, how they are
currently defined, what the evidence is that supports these ideas, how these ideas stand in
relation to one another—so that they can intentionally ask both “good” design mode and belief
mode questions (Bereiter & Scardamalia, 2003, 2007). Bereiter and Scardamalia (2007) indicate:
Good questions to ask in belief mode are:
· What does this statement mean?
· Is it true?
· What’s the evidence?
· What are the arguments for and against?
Important questions to ask in design mode are:
· What is this idea good for?
· What does it do and fail to do?
· Does it have a future?
· How could it be improved? (pp. 25-26)
Work in belief mode and design mode is important in continuing medical education. Belief mode
work emphasizes truth, evidence, and personal knowledge. Doctors deal with belief issues in
terms of assessing information quality, which allows for consideration both of degree of certainty
and of the consequences of being wrong. They need to know the most current science, facts and
evidence, treatment and management to perform best-practice care. Design mode work deals
with issues of usefulness, adequacy, fruitfulness, and improvability (Bereiter & Scardamalia,
2003, 2007). The objectives are idea advancement and knowledge creation. Continuing medical
education needs to also address improvement in personal and community practice, locally and
world-wide. Design mode work requires a different type of thinking around different questions
than belief mode work. The problem space must enable both modes and ideally create synergy
between them.
2.3.5 The Knowledge Building Problem Space
Open and ill-defined problem spaces provide opportunities for emergence and self-organization
(Scardamalia & Bereiter, 2006) and represent distinguishing features of design-mode
environments. A technically unstructured problem space such as Knowledge Forum can provide
collective opportunities for testing understandings, identifying issues, and advancing concerns,
and can also provide challenges for introducing structure to problematize an issue or clinical case
contributed by participants, the facilitator, or the curriculum designer.
Knowledge Building in Continuing Medical Education … 33
The Knowledge Building/Knowledge Forum problem space in this study is both structured and
unstructured. It is structured by a clinical case and learning objectives that presuppose discourse
structures of case-based reasoning (Kolodner, 1992) and problem-centred knowledge (Bereiter,
Burtis, & Scardamalia, 1994); it is also unstructured, in that the case is used as a springboard,
intended to open discussion, not close it down.
Structures vary across contexts, communities, and cultures and are determined by participant
interactions. Facilitator/student participant structures that are more symmetric have been found to
shift power and responsibility to students resulting in greater agency and authority (Tabak &
Baumgartner, 2004).
Knowledge Building student-teacher participant structures have not received much attention in
the research literature. Important studies in a grade-school context in Knowledge Forum
provided interesting results on teacher-student question and exchange patterns (Zhang,
Scardamalia, Reeve, & Messina, 2009; Zhang, Scardamalia, Lamon, Messina, & Reeve, 2007,
Zhang et al., 2005). Typical patterns like those found by Cazden (1988) characterized by teacher-
initiated questions followed by student responses were found, as well as those that went beyond
this structure by building on students’ ideas. Details on studies involving participant structures
and Knowledge Building facilitation will be provided later in this review of the literature. These
studies were used to inform ideas and methods for this research study.
2.3.6. Participant Structures and the Codesign of Knowledge Building
2.3.6.1 Facilitating Knowledge Building. Ng & Law (2003) found that a facilitator’s ability to
scaffold online Knowledge Building was particularly important when the subject matter context
was beyond the students’ zone of proximal development. In another study with young students,
Viilo, Seitamaa-Hakkarainen, and Hakkarainen (2011) analyzed a teacher’s weekly reflections
on facilitating knowledge building online and indicated that “the essential question, when
analyzing teacher’s support for collaborative inquiry project, is who is carrying the responsibility
for advancement of the process and who is posing the questions to be asked” (p. 51). They found
the teacher had a crucial influence in the process and recommended further research to cultivate
models and ideas of social cultural practices of knowledge building facilitation and how these
practices are transformed by technology.
Knowledge Building in Continuing Medical Education … 34
Hmelo-Silver and Barrows (2008) examined face-to-face facilitation of knowledge building with
second-year medical students in a problem-based learning group. They found that the facilitator
supported knowledge building by “asking open-ended metacognitive questions and catalyzing
group progress. The students took responsibility for advancing the group’s understanding as they
asked many high-level questions and built on each other’s thinking to construct collaborative
explanations.” Hmelo-Silver and Barrows explained that helping facilitators and students ask the
right kind of questions to provoke building on each other’s thinking may be a key to
orchestrating knowledge building discourse.
Based on the strength and findings of these studies, the study herein will examine social network
structural and semantic analysis primarily of build-on notes and focus on the relationship of
questions to participant structures and Knowledge Building.
2.3.6.2 Facilitating knowledge building in relationship to coaching reflection-in-action.
Similarities exist between the descriptions of knowledge building facilitation in an undergraduate
medical education problem-based learning seminar (Hmelo-Silver & Barrows, 2008) and the role
of professor as coach in an architectural design studio setting (Schön, 1983). The dialogue
between coach and student and the paradoxes of learning to design highlight relationships
between reflection-in-action and knowledge building. These relationships are worth considering;
however, the key feature of knowledge building as a social cultural practice remains unaddressed
in Schon notion of reflection-in-action, which he describes as dialogical, between coach and
student and personal (Schön, 1983, 1987).
Schön (1987) focused on the need for artistry in professional education and for future practice;
his work has been influential in continuing medical education: “The studio tradition of design
education is consistent with an older and broader tradition of educational thought and practice,
according to which the most important things—artistry, wisdom, virtue—can only be learned for
oneself” (p. 84). He advocated for participatory, reflective practice.
Schön (1987) indicated the knowledge that one needs to know “will not come from teaching but
from questioning” ( p. 85). Like the knowledge building studies referred to previously,
questioning is seen as central to the process of facilitating improvement.
Knowledge Building in Continuing Medical Education … 35
Schön (1987) reframed teaching, quoting Carl Rogers, who believed that “the most important
things cannot be taught but must be discovered and appropriated for oneself. . . . [One] attributes
to [oneself] and others a capacity for self-discovery and functions as a paradoxical teacher”
giving central importance to one’s own role as a learner (p. 92). The importance of Knowledge
Building symmetry and relationship to agency has been discussed.
Overcoming the “paradoxes and predicaments in learning to design” depends on the capacity of
student and studio master to communicate effectively with each other, in spite of the potential for
vagueness, ambiguity, or obscurity inherent in the things about which they try to communicate.
Their search for convergence of meaning,” as in knowledge building, is what is at stake (Schön,
1987, p. 99).
Schön called the process of designing a “reflective conversation with the situation.” A brief
review here of the designer’s personal reflection-in-action sets the stage for how the dialogue
between the coach and student can further reflective processes. Here again we note two levels:
the designer’s reflection with the situation, and with the coach’s reflections on the designer’s
reflections. In this way, the role of teacher becomes that of colearner. Reflection on situation is
active:
[The designer]. . . works in particular situation, uses particular materials, and employs a
distinctive medium and language. Typically, his making process is complex. There are
more variables - kinds of possible moves, norms, and interrelationships of these - than
can be represented in a finite model. Because of this complexity, the designer’s moves
tend, happily or unhappily, to produce consequences other than those intended. When this
happens, the designer may take account of the unintended changes he has made in the
situation by forming new appreciations and understandings and by making new moves.
He shapes the situation, in accordance with his initial appreciation of it, the situation
“talks back,” and he responds to the situation’s back-talk.
In a good process of design, this conversation with the situation is reflective. In
answer to the situation’s back-talk, the designer reflects-in-action on the construction of
the problem, the strategies of action, or the model of the phenomena, which have been
implicit in his moves. (Schön, 1983, p. 79)
“Three dimensions of this process are particularly noteworthy: the domains of language in which
the designer describes and appreciates the consequences of his moves, the implications he
discovers and follows, and his changing stance toward the situation with which he converses”
(Schön, 1983, p. 95).
Knowledge Building in Continuing Medical Education … 36
The implications of the designer’s moves come at “choice-points” (Schön, 1983, p. 99). The
designer considers not only the present choice but also a tree of further choices. After
consideration of many of a few “what if” situations the designer must make a binding choice that
has implications for all further moves. Thus there is a continually evolving web of implications
within which the designer reflects-in-action. As the designer spins out his web of moves, a
continual shift in stance occurs. The designer must oscillate between the unit and the total,
between involvement and detachment.
Now that we have considered the designer’s personal reflection-in-action with the situation
encountered with the artwork, we now consider how the dialogue between the coach and student
can further this process. “Questioning, answering, advising, listening, demonstrating, observing,
imitating, criticizing—all are chained together so that one intervention or response can trigger or
build on another” (Schön, 1987, p. 114). The important notion of building on ideas, as in
knowledge building, is evident here. Schön (1987) likened the chain of reciprocal reactions and
reflections of the student and coach to a ladder where the “higher levels of activity are “meta” to
those below. The levels of action and reflection on action can be seen as the rungs of a ladder.
Climbing up the ladder, one makes what has happened at the rung below into an object of
reflection” (Schön, 1987, p. 114).
It is noteworthy that facilitation of cognitive design work in the art studio is similar in some
aspects to facilitating knowledge building. However, there are three key differences: knowledge
building social cultural practices focus on collective advancement as opposed to teacher-student
dyads; knowledge work is intentional which includes reflection early in the process; and artifacts
for knowledge work are objectified. This latter includes reflections, which in Schön’s world
often remained located within one’s mind, not objectified for further work.
Similar to Schön’s (1983) articulation of four steps of reflection, Mamede and Schmidt’s (2004)
five-factor model of reflective practice in medicine consisted of “deliberate induction; deliberate
deduction; testing and synthesizing; openness for reflection; and metareasoning.” Mamede and
Schmidt’s framework advocated a meta level that does not seem attainable in isolated individual
work to the same degree as community knowledge building.
Knowledge Building in Continuing Medical Education … 37
2.3.6.3 Expertise and social networks for cocreation. Bereiter and Scardamalia (1993)
addressed the issue of the development of expertise in their book, Surpassing Ourselves: An
Inquiry Into the Nature and Implications of Expertise. They distinguished between “expert-like
and non-expert-like learners” and defined expert-like learners as those who are engaged in
progressive problem-solving through distributed knowledge building, and working on communal
artifacts.
Regehr and Norman (1996) stated that expertise is characterized by the development of elaborate
semantic networks and that true understanding is not defined by the quantity of information a
person possesses but by the extent to which this information is organized into a coherent,
mutually supportive network of concepts and examples. The semantic network is viewed as an
elaborate set of connections between abstract concepts and/or specific experiences, and the links
are viewed as linkages between concepts and experiences that are based on meaning. The issue
here once again is that this web of knowledge is located inside one’s mind.
Yet the notion that meaning and understanding is more than memorization of facts is highly
relevant to continuing medical education. Meaning is enhanced through activation of relevant
prior knowledge that facilitates processing of new information and problem discussion stimulates
activation and elaboration of prior knowledge that subsequently facilitates processing and
comprehension of new information and retrieval (Norman & Schmidt, 1992; Regehr & Norman,
1996; Schmidt, Norman, & Boshuizen, 1990). Importantly for this study, Regehr and Norman
(1996) indicated that elaboration is the process of considering a piece of knowledge in a richer,
wider context. Strategies of elaboration are discussion, writing, answering questions, or using the
knowledge to understand a problem (Norman & Schmidt, 1992; Regehr & Norman, 1996).
It is evident how Regehr and Norman’s (1996) more traditional cognitive-psychology research
on personal semantic networks, within one’s mind, may contribute to an understanding of larger-
community. What is missing is the importance of objectified knowledge work aimed at
intentionally advancing ideas of importance to the broader community, the social network. This
is Knowledge Building collective responsibility of professional practice and expertise. This
distinguishing feature of Knowledge Building can be described using social network analyses
and tools in Knowledge Forum.
Knowledge Building in Continuing Medical Education … 38
2.3.7 Knowledge Forum Suite of Analytic and Social Network Tools to Support
Knowledge Building
Knowledge Forum is second-generation CSILE. It incorporates social network and semantic
analysis tools. Using social networks to analyze patterns is a common research and evaluation
strategy, and has a history in the learning sciences, and particularly within the knowledge
building research community (Hewitt; 2005; Hewitt & Brett, 2007; Hewitt & Teplovs, 1999;
Zhang et al., 2005).
Salomon (1994a, 1994b) and Chi (1997) used mapping techniques to analyze difference in
patterns in participation and interaction. Chi (1997) presented a “practical guide” on how to
quantify qualitative analysis. She indicated that by mapping the data, one can begin to seek
patterns in the results, and these patterns can be statistically confirmed. In a study of children and
dinosaurs, Chi and Koeske (1983) found a correlation between density of links in a cluster and
better organization and coherence ideas, when compared to a more diffuse cluster structure, even
though the total number of links and nodes used to represent the two structures were identical
(Chi, 1997, p. 299).
Chi (1997) said:
In general, it does not appear difficult to quantify the density of graphical representations
of coded data . . . one should not have to rely on subjective visual assessments of the
structure of graphical representations. . . . Quantifying a perceived pattern in a graphic
representation of coded data is not the same thing as quantifying the total number of
elements in it. (p. 300)
It is important to note that the quantification occurred at the level of if the pattern, which
attempts to capture the structures in the representation.
Interpretation of perceived patterns in data and checking the validity of interpretation is key. Chi
(1997) noted interpretation depends entirely on the hypotheses being tested, the research
questions asked and the theoretical orientation of the investigator. Interpretation of data can be
performed in terms of strategies and processes, or structure and content of knowledge or both.
She cautions that the interpretation of patterns in data is often more persuasive if there are other
converging evidence or analyses.
Knowledge Building in Continuing Medical Education … 39
Knowledge Building researchers not only looked at patterns of participation, but advanced this
idea by adding other dimensions, such as examining growth in conference threads (Hewitt &
Teplovs, 1999), why threads of discourse die in asynchronous environments (Hewitt, 2005), and
relationships between number of participants and online activity patterns (Hewitt & Brett, 2007).
Early notions of social network analysis were embedded in the Knowledge Forum analytic
toolkit (ATK; Burtis, 2001). The ATK could almost instantly calculate who built on whose notes
in a Knowledge Forum view and provide a numeric grid of results. The names of all participants
where listed on the x and y axes and a matrix of numbers was presented indicating the number of
times one person built on another’s note or notes. This visual representation was difficult to
make sense of and did not easily facilitate pattern recognition. Even more perplexing and
challenging were comparisons across time. Limitations in representation of data and difficulty of
interpretation led this researcher not to include these data in this study.
New Knowledge Forum Social Network Visualization tools (Teplovs, 2010) enable us to see the
network of connections, analyze the relationships between connections, and over time see shifts
is patterns, that may signify shifts in knowledge building or belief mode to design mode
thinking. In Teplovs’s (2010) doctoral thesis study he used notes as a proxy for ideas. In essence
the network visualization was not just of connected notes, but a network of ideas. He stated that
although notes do not correspond exactly with idea, most notes contain at least one or more
ideas, and therefore are a convenient unit of analysis. Teplovs created a systems architecture, in
particular a visualization layer that he describes lies above the discourse layer. The “Knowledge
Space Visualizer (KSV) . . . is a prototypic system for showing reconstructed representations of
discourse-based artifacts and facilitating assessment in light of patterns of interactivity of
participants and their ideas” (Teplovs, 2010, p. 1). The Knowledge Space Visualizer “uses Latent
Semantic Analysis to extend techniques from social network analyses, making it possible to infer
relationships among note contents,” (Teplovs, 2010, p. 1), thereby enabling the study of idea
networks in conjunction with social networks in online discourse.
The Knowledge Space Visualizer provided quantitative network metrics of density and degree,
along with the visualizations. It provided new ways to study relationships between notes, to
examine changes over time (not improvability of ideas), and to support assessment—for
embedded, concurrent feedback (as in Teplovs’s thesis) and for research-based innovation. This
Knowledge Building in Continuing Medical Education … 40
study makes use of the Knowledge Forum network analytic tools of density and degree. These
new tools were important to the current study.
A number of recent conference papers and workshops have disseminated new ideas about
visualization assessments of Knowledge Building that are highly relevant to advance research.
These include identification of idea clusters (Teplovs & Fujita, 2009), use of latent semantic
analysis and term clouds (Teplovs & Fujita, 2009), pinpointing of pivotal moments (Fujita &
Teplovs, 2009), and visualization of group cognition using semantic network analysis (Sha,
Teplovs, & van Aalst, 2010). These types of advances are key to developing new tools to assess
Knowledge Building and collective innovation. The next section will explore the use of social
network analysis.
2.4 Cluster 3: Social Network Measures and Sociocognitive Dynamics
That Enable Work Over and Above Traditional Learning
Work beyond best practices requires work beyond traditional assessment, which in turn requires
new tools to answer new questions and investigate new perspectives. In this section I review
literature on network measures and sociocognitive dynamics to explore knowledge work focused
on collective outcomes and over and above traditional learning.
2.4.1 Social Network Analysis in the Social Sciences
Engeström, Miettinen, and Punamäki (1999) provided a framework in which to understand
activity theory from both individual and social perspectives, as a system mediated by artifacts
through “expansive” cycles of internalization and externalization for transformation. Engeström
et al. (1999) indicated that “Key findings and outcomes of such research are novel activity-
specific, intermediate-level theoretical concepts and methods—intellectual tools for reflective
mastery of practice” (p. 36). The concept of examining activity as a principle of explanation is
aimed at identifying potential for qualitative change at the local level of activity, as well as at a
grander scale:
The mightiest, most impersonal societal structures can be seen as consisting of local
activities carried out by concrete human beings with the help of mediating artifacts even
if they may take place in high political offices and corporate boardrooms instead of
factory floors and street corners. In this sense, it might be useful to try to look at the
society more as a (layered network of interconnected activity systems and less as a
Knowledge Building in Continuing Medical Education … 41
pyramid of rigid structures dependent on a single centre of power. (Engeström et al.,
1999, p. 36)
Engeström’s (1999) ideas have been highly influential in social psychology and educational
technology research. However, the roots of social network analysis are often attributed to Linton
C. Freeman (1978/1979, 2004), a social scientist and mathematician.
Freeman’s (1978/79) analysis of structural centrality in social networks provided us with three
measures: one absolute; one relative, a measure of centrality of position in a network; and one
reflecting the degree of centralization of the entire network. Graph theory and point measures
were calculated at 34 levels of centrality. Overall, Freeman’s network centrality measures
indicated that the star or wheel pattern is assigned the maximum centrality score and the circle
the minimum score: “between these extremes . . . we are faced with an embarrassment of
intellectual riches. We have not one, but three conceptions of centrality, and we have a family of
measures for each” (Freeman, 1978/79). Thus centrality can affect group processes and be
interpreted in three ways. For example, if we propose that leadership depends on centrality, then
we need to know if this means centrality as control, centrality as independence, or centrality as
activity. Any one or a combination of them may be appropriate in an application such that within
this study.
Other social network scientists have contributed to ideas, measurements, and interpretations of
these measures more recently. Barry Wellman (1983) has done much to clarify basic principles
of network analysis. He has reframed our thinking in this field by pointing out that the world is
composed of networks, not groups (Wellman, 1988). Marin and Wellman’s (2011) recently
published introduction to social network analysis is particularly helpful in setting forth guiding
principles of network analysis that direct our attention to the fact that this is research based on
relations, not attributes.
Social network analysis has flourished in the social sciences in recent years and consequently
many books and articles have been published illuminating methods of data collection,
mathematical representations of social network, structural properties, such as
centrality/prestige/prominence, group cohesion (including cores and cliques), positional analysis
(including roles and ties between and within positions), and block measures, such as network
density (Carrington, Scott, & Wasserman, 2005; Scott, 2009;Wasserman & Faust, 1994).
Knowledge Building in Continuing Medical Education … 42
Even more recently, along with publications on social network structural analysis methods, came
publication of more conceptual works and results of research studies. Current issues, methods,
and a breadth of research reports from many different societal aspects are covered in the 2011
publication of The Sage Handbook of Social Network Analysis (Scott & Carrington, 2011).
However, this extensive volume does not include studies related to health care, medicine, or any
aspect of medical education. Another recent publication, Social Networks and Health: Models,
Methods, and Applications (Valente, 2010), fills this gap and specifically targets the heath care
audience, but not health sciences education. Despite this, helpful connections are made between
network analysis outcomes and applications to identification of opinion leaders, key players,
bridge participants and group characteristics important to health care.
In a more conceptual vein, power structures of social networks are explored by John Scott
(2001), in his book entitled Power. Scott’s writing was helpful in framing this research study,
particularly in terms of his work on expertise and professionalism and authoritative structures; he
links his ideas to Foucault previous work and establishes ways of interpreting power analyses.
Social network power relationships have also been analyzed within the classroom; one
particularly important study is reported in the next section. Other sections will review results of
studies pertaining to various aspects of social networks within different environments, face-to-
face and online.
2.4.2 Social Network Analysis and Power Structures in Education
Four important studies are reviewed in the next section. They are separated from others below in
the education section, since they were highly influential in the conceptual development and
methodological design of this study.
2.4.2.1 Power structures concepts. Cornelius and Herrenkohl (2004) used interviews to
examine power structures in Grades 5 and 6 classrooms and found that with the introduction of
new technology, new participant structures emerged, changing relationships between students
and the teachers, amongst students themselves, and their relationship with the learning concepts.
Social network analysis of changes in social interactions and participant structures clarified “the
way that these structures enable and incite disciplinary thinking.” The emergence of new
participant structures transforms power and authority:
Knowledge Building in Continuing Medical Education … 43
Further we believe that looking at participant structures in terms of power allows an
effective means for analyzing why some structures are more successful than others, whey
certain students may “appropriate” the structures more readily than others, and why some
structures may be better suited to certain disciplines than others. (Cornelius &
Herrenkohl, 2004, p. 468)
The authors framed participant structures within a notion of cultural tools and explained that by
doing so it was their intention to move beyond behavioural explanations of the individual, and to
focus on relationships.
Similarly, Cornelius and Herrenkohl (2004) framed the concept of power in terms of Foucault’s
(2001) definition; the nature of power is seen as ‘“strictly relational” and as containing “many
points of resistance”(p. 477). In an educational setting, this means that manifestations of power
could be found in any interaction or relationship. Power is not seen as external to learning, nor is
it seen as imposed from the top down; it is seen as inherent to groups and individuals. Power is
not seen as stable; it is seen as dynamic, continually changing. In Cornelius and Herrenkohl’s
(2004) conceptualization, “relationships of power as existing on a balance scale, with situation
factors causing the positions of persons in an environment to constantly shift and change with the
potential of being tipped in different directions” (p. 469). A sociocultural approach to the
examination of power is taken in this study. They indicate a new vocabulary is required to
explicate power relationships located in interactions and for conceptualizing dynamic ways in
which people, mediating artifacts, and sociocultural tools, such as new participant structures
influence each other.
Results of the current study indicate that three conceptualizations of power relate to classrooms
and education: ownership of ideas, whether that be student or teacher; partisanship, which
describes relationships among students and their interaction with concepts and with other; and
persuasive discourse, which relates to ways in which ideas are communicated, as authoritative or
internally persuasive (Cornelius & Herrenkohl, 2004). Additionally, each of these
conceptualizations reflects different and corresponding relationships that exist in the classrooms:
between students and concepts; among students; and between students and teachers. Results
indicated ownership of ideas narrowed the distance between students and the scientific concepts;
students created their own theories and used ideas flexibly. Partisanship enabled them to
collectively explain ideas; and, persuasive discourse allowed them to try to convince their peers
by providing proof for their arguments to support their theories.
Knowledge Building in Continuing Medical Education … 44
Cornelius and Herrenkohl’s study (2004) also indicated that the teacher’s role was central to
students’ accomplishment through four main structures: (a) problematizing content, through
building-on prior knowledge; (b) questioning, and challenging ideas; (c) giving students
authority, recognizing students’ ideas and contributions, and holding students accountable; and
(d) providing relevant resources. Discourse sequences were found to be very different from
Cazden’s (1988) I-R-E sequence, in which the teacher initiates most questions, students respond,
and then the teacher evaluates. Teachers play an important role in shaping student participation,
by highly scaffolding interactions and creating opportunistic power relations (Cornelius &
Herrenkohl, 2004). The lens of power can provide new perspectives and insights into the
strengths and weaknesses of participant structures in the classroom. This analytic lens also
allows for new explanations of the social networks in education, such as how students participate
or collaborate. Position analysis of power is used in the current study.
2.4.2.2 Social network analysis of roles and power structures. In another study, Aviv,
Erlich, Ravid, and Geva (2003) identified roles and power structures through social network
analysis of cohesion and cliques. They used mixed-methods, combining social network analysis
with content analysis, to examine two asynchronous learning environments, in two university-
level business ethics courses. One was structured and a formal part of a course and the other was
considered unstructured and not a required part of the course. Analysis of power or centrality
was measured by Eigenvector centrality, and Freeman’s in-degree and out-degree centrality. In
additional network density was calculated and compared. They found that the structured design
was associated with a high degree of knowledge construction. Many cohesive cliques developed.
Students took on bridging and triggering roles and the tutor held relatively little power, but
closely monitored the discourse. Students belonging to more than one clique often perform
bridging roles in that they facilitate the flow of information. In the non-structured environment
activity was lower; there were fewer cliques and most of the students were passive, following the
teacher who held a central position of power. Social network measures used in this study were
used in the current study.
2.4.2.3 Social network position and identity. Ligorio (2009) explored the relationship
between knowledge building and identity building. She indicated, in contrast to Lave and
Wenger (1991), that identity in a community of practice (Wenger, 1998) is not always developed
through a linear movement from the periphery to the centre; Ligorio found that, in fact, this
Knowledge Building in Continuing Medical Education … 45
process could described by three trajectories in addition to Lave and Wenger’s original one.
These four trajectories are relevant to the current study:
Stability. Some students tend to maintain the same level of centrality over time.
Progressive centralization. This is a linear trajectory, from the periphery toward
increasing centrality, as described by Lave and Wenger (1991).
Progressive decentralization. This indicates an inverse linear trajectory to that above,
towards an ever-diminishing centrality.
Non-linear stability. This characterizes the trajectory of decentralization in the middle of
the discussion, before adopting a central position at the end of the discussion; or,
conversely, central position in the middle of the discussion, before decentralizing.
(Ligorio, 2009, p. 40)
Ligorio (2009) concluded that these different trajectories influence the structure of the online
community and that students who develop a stabilized trajectory of centrality become a
“referential nucleus” within the community for others to build around (p. 40). Ligorio (2009,
2010) put forward the notion of “digital I-positioning” aimed at personal change and indicates
that a “community is transformed on the basis of the interactions between positions that take
place simultaneously on multiple levels” (Ligorio, 2009, p. 41), including the individual,
interpersonal, and community levels. Development of self and sense-making through
examination I-positioning sheds interesting new light relevant to professional development along
trajectories of lifelong learning.
2.4.2.4 Social network analysis of facilitator roles and participant structures. Tabak and
Baumgartner (2004) explored teacher-student participant structures focusing on analysis of the
teacher in achieving balance between authoritative and persuasive discourse with science
students (face-to-face in a classroom). They demonstrated how the mastery of the cultural tool
occurred through symmetric student-teacher interactions in a participant structure they called
partner. Although it was rare in comparison to other observed structures, mentor and monitor,
they determined that symmetrical partner structures can provide profitable opportunities for
knowledge work. They indicated that “the partner participant structure is similar to the mentor
participant structure, but the main difference lies in the posture that the teacher assumes (Tabak
& Baumgartner, 2004). In the partner structure, the teacher and students are said to share the
same role of investigator; in the mentor structure the teacher is an outsider and scaffolds the
students to align their discourse with the norms of study. Tabak and Baumgartner concluded:
Knowledge Building in Continuing Medical Education … 46
“Rather than representing opposing or competing instructional forms, the mentor and partner are
efficacious as an ensemble. . . . Balance is achieved between authoritative and persuasive
discourse” (p. 424). The notion of teacher-as-partner, as opposed to coach or guide-on- the-side,
holds particular promise. Tabak and Baumgartner suggested that it is possible to promote
rigorous subject-matter learning typically attributed to formal learning environments as well as
the symmetry and personal outcomes typically attributed to informal learning environments. I
conducted qualitative, within-note analysis of participant structures using the modified categories
from the Tabak and Baumgartner study. My study found a similar efficacious ensemble in online
facilitation—balance between the mentor and partner and expert stance was readily apparent.
2.4.3 Social Network Analysis of Sociocognitive Dynamics in Knowledge Building
In the last 10 years, social network analysis has been conducted in different educational
environments; published results are interesting and varied and range from grade school students
to higher education. No published social network studies were found in the context of continuing
medical education. However, information on the Internet was found on two studies in process
with undergraduate medical students and one with physicians in practice.
2.4.3.1 Social network analysis of Knowledge Building in K-12 classrooms. Sha and van
Aalst (2003), Philip (2010), and Moss and Beatty (2010) have employed social network analysis
of discourse in Knowledge Forum.
An early study by Sha and van Aalst (2003) aimed to probe collaboration at a more systemic
level. In two case studies (one with a Grade 4 class and one with a Grade 9 class) they employed
Knowledge Forum analytic toolkit measures, such as Who’s read whose note?, Who’s built on
whom?, number of notes created, percentage of notes linked, and percentage of notes read; and
social network analytical measures of in-degree, out-degree, and betweenness. Although there
were no significant correlations between the social network analysis variables and the use of
Knowledge Forum features in this study, the methods are emphasized over the results. The point
being that meaningful and readily understandable collective assessment measures are required
for research and classroom assessment of socially constructed knowledge and understanding.
Individual activity measures cannot capture this aspect.
Knowledge Building in Continuing Medical Education … 47
In another study by Philip (2010), student interaction patterns were examined in a Grade 5/6
Knowledge Building class. In this study Philip used a series of similar Knowledge Forum
analytic toolkit and social network analysis measures, as well as calculating network density.
Philip noted the relevance of Social Network density and centrality measures in combination
with more traditional analytic toolkit measures for new ways of revealing group processes of
collective knowledge building. Sociogram visualizations provide a quick tool for feedback on
collective participation.
Moss and Beatty (2010) used social network analysis visualizations and note content analysis to
demonstrate patterns consistent with the knowledge-building principle of democratizing
knowledge. Grade 4 students at all achievement levels participated equally and with agency
supported by an egalitarian online culture.
2.4.3.2 Social network analysis of knowledge building in higher education. Lipponen,
Rahikainen, Lallimo, and Hakkarainen (2003) used similar mixed-methods to analyze patterns of
asynchronous online participation and quality of discourse using content analysis in a course
with university level students. Quality of discourse was found to be relatively weak, although
participation and network density was strong. Thus measures of density can be misleading; there
are few comparatives to understand what is optimal density for high-quality learning. Numerous
postings of unrelated information can lead to information overload that doesn’t relate to
knowledge improvement. Therefore the missing component seems to be intentional knowledge
building or epistemic agency.
Some more recent studies have employed social network analysis for embedded concurrent
assessment of knowledge building (Wang & Li, 2007). Others have used social network position
analysis to determine how thought leaders, as core participants, interacted with their peers and
impact knowledge building (Waters, 2008). Most interestingly Lu Wang (2010) analyzed two
online university level courses in terms of core periphery structures and determined these
networks contained “structural holes” (Burt, 2004). Opinion leaders positioned at structural holes
exhibited different characteristics in terms of knowledge building from those positioned at the
core of the network.
Knowledge Building in Continuing Medical Education … 48
The importance of structural holes and their relationship to emergent “good ideas” was first
described by Burt (2004) in the business literature. He said: “people who stand near the holes in
a social structure are at higher risk of having good ideas” (Burt, 2004, p. 349). He argued that
opinions and ideas within groups are more homogeneous than between groups. Therefore people
connected across groups, are potentially exposed to alternative ideas and different ways of
thinking, which in turn, affords greater possibilities of options and emergence of new ideas.
“New ideas emerge from selection and synthesis across the structural holes between groups.
Some fraction of those new ideas are good. . . . A good idea broadly will be understood to be one
that people praise and value” (Burt, 2004, p. 349). In essence, Burt indicated, people whose
networks spanned structural holes can become brokers of ideas and brokerage across structural
holes between groups is the mechanism by which social capital increases. This idea has
important implication for large-scale Knowledge Building studies with multiple interacting
groups.
2.4.3.3 Social network analysis of an international knowledge building network. An
important design research study examined the development and international spread of
Knowledge Building innovations through the Knowledge Society Network in Knowledge Forum
(Hong, Scardamalia, & Zhang, 2010). The Knowledge Society Network was examined in terms
of social interaction and idea interaction through analysis of contribution, shared problem space
and multiple problem spaces.
Cumulative growth (e.g., number of participants, readers), a series of matrix correlations based
on participant interaction patterns, and calculation of network degree of centralization (core
group, periphery group) are expressed both quantitatively and qualitatively for participant
interaction in a shared problem space. Idea interaction patterns, are also expressed quantitatively
and visually, through network of patterns of idea interaction of weak/strong-tie comparisons,
based on descriptions by Granovetter (1983) and Haythornthwaite (2002).
Hong et al. (2010) offered the following definition: “The degree of participation in SNA means
the total number of connections between a participant and other participants in a network; and
degree centralization is the mean degree number (Freeman, 1979).”
Knowledge Building in Continuing Medical Education … 49
Although these quantitative and visual analyses successfully illustrated growth over time and
intensity (week/strong) participant and idea interaction, researchers indicated that these analyses
could not explain more subtle, metarelationships between participant and idea interaction— “for
instance, how ideas might interact with each other, within the weak, peripheral participation
network” (Hong et al., 2010, p. 8). The following four subnetworks were analyzed:
(a) Sustained Knowledge Innovation Network—with strong participant and idea
interaction; (b) Emerging Network—with weak participant and idea interaction; (c)
Intensive Participant Interaction Network—with strong participant interaction but weak
idea interaction; and (d) Frequent Idea Interaction Network—with weak participant
interaction but strong idea interaction. . . . Each subnetwork represents a type of network
dynamic, and it is interesting that all four of these dynamics are evident in KSN. (Hong et
al., 2010)
It is also interesting that the interpretation of these analyses can be considered to be assessment
of democratization of knowledge building participation, processes, interactions, and ideas.
Outcomes successfully identified strong participant and idea interaction, as well as isolated
subcommunities in the periphery and temporally distant Knowledge Forum views that can be
enhanced.
2.4.4 Social Network Analysis in Medicine and Medical Education
Social network analysis has been used in public health studies, research in medical education,
and in medical practice, to create information networks. Only one pilot study was found that
examined Knowledge Building in a continuing medical education environment (Punja, 2007).
No studies were found that focus on social networks relationships of Knowledge Building in a
continuing medical education course.
Social network analysis has brought to the fore new relationships important to public health
information. For example, Christakis and Fowler popularized the notion of social networks
through their studies on the spread of obesity (2007), smoking (2008), and happiness (Fowler
and Christakis, 2008). In another study, by Gardy and colleagues (2011), whole-genome
sequencing results were overlaid on social network analysis to trace a tuberculosis outbreak and
describe the outbreak dynamics at a higher resolution.
Recently, in undergraduate medical education research, Walton and Oswald (2011) examined
patterns of interaction in a second-year problem-based learning group at the University of
Knowledge Building in Continuing Medical Education … 50
Alberta, Canada. They used Freeman’s normalized centralization measures to visually convey
differences between a highly interactive group and a less interactive group. They also used
graphical measures to convey percent of individual participation (verbal contributions to
discussion). In both groups they found that the tutor spoke more than 50% of the time. These
results could be used as formative feedback tools aimed at improving student responsibility and
contributions to the social network.
In another study, currently underway at the University of Ottawa’s Faculty of Medicine,
(Sutherland et al., 2011), social network analysis is being used to determine if is possible to
measure student competencies such as collaboration and other behaviours identified in the
CanMEDS roles. This longitudinal study will follow students from 2008 (first year) to 2012
(medical clerkship). An outcomes report is accessible online, as presented by Dr. Stephanie
Sutherland, at University of Ottawa, Academy for Innovation in Medical Education Rounds
(Sutherland, 2011). Characteristics such as network size, interconnectivity, and brokerage were
evaluated and reported to students.
Social network analysis has been used in the context of continuing medical education to help
identify and connect physicians with similar practice and personal characteristics with each other
to create informal and formal networks of learning, to facilitate information flow, and improve
accountability in health care organizations (Miller et al., 2008). In this vein, Keating, Ayanian,
Cleary, & Marsden (2007) used social network analysis to evaluate the spread of influential
discussions among primary care physicians in a hospital-based practice.
2.4.5 Complexity of Social Network Discourse: Visual Representation, Elaboration, and
Identification of Misconceptions
Knowledge Forum enables visualization of individual thinking and collective understanding.
Making visually explicit one’s knowledge, ideas, and understanding—or lack thereof, is a
foundational premise of Knowledge Building. The importance of working with “objectified
knowledge” (Bereiter, 2002c), putting “ideas-at-the centre” (Scardamalia, 1999), creating
“awareness of knowledge lacks” (Bereiter & Scardamalia, 1989) and misconceptions (Burtis,
Chan, Hewitt, Scardamalia, & Bereiter, 1993) have long been a part of the literature on
Knowledge Building.
Knowledge Building in Continuing Medical Education … 51
“Making thinking visible” (Collins, Brown, & Holum, 1991, p. 1), making knowledge tacit
(Polyani, 1983) as a knowledge object (Popper, 1972) for collective design mode work is a
radical shift from traditional individual learning where cognitive processes, reasoning, and
misconceptions often remain invisible—in one’s head (Bereiter & Scardamalia, 1989). Some
misconceptions have been shown to be robust and are of general concern in education (Chi,
2005) and of particular concern in medical education (Lester & Tritter, 2001) and continuing
medical education (Hanna, Premi, & Turnbull, 2000). Misconceptions in online learning pose
additional concerns of propagation (Burtis et al., 1993). These issues are addressed through
collective as opposed to individual online learning. The benefits of making ideas explicit and
working collectively with knowledge become evident in this next series of studies.
Accuracy of clinical reasoning, identification of misconceptions, and correction of errors are key
concerns in at all levels of medical education and practice. Elstein (1999) examined heuristics
and biases as sources of error in clinical reasoning. Earlier work by Custers, Regehr, and Norman
(1996) reviewed mental representations of medical diagnosis, i.e. prototypical, instance-based,
and semantic networks, including schema and script models. The work by Bordage and
colleagues (1991,1994, 1997) on semantic networks is particularly relevant to the Knowledge
Building context of this study and lends further insight and a framework for analysis of
complexity of discourse that will be used herein.
Bordage (1994) specified and classified the organization of four types of clinical discourse:
reduced, dispersed, elaborated, and complied. He determined that elaborated structure is
associated with accurate resolution of complex problems, that is, 75% to 80% resolution for
elaborated discourses as opposed to near-zero resolution for dispersed discourses. Bordage noted
that by making the semantic connections more visible and conscious—for example, using them
when reading textbooks or during case presentations and discussions—students could learn to
solve a problem by defining it clearly first before blindly generating lists of diagnostic
impressions. Bordage distinguished between simply listing potential diagnosis, as opposed to
articulating differential diagnoses with appropriate justifications and elaboration on reasoning,
before at making a commitment to a working diagnosis. As the title of this 1994 paper indicates,
he argued for “Elaborated Knowledge: A Key to Successful Diagnostic Thinking.” Bordage,
Connell, Chang, Gecht, and Sinacore (1997) created and validated a framework for semantic
content analysis of clinical case presentations.
Knowledge Building in Continuing Medical Education … 52
In a later study, Chang, Bordage, and Connell (1998) focused on problem representation, the first
step in problem solving. They found striking differences between successful and unsuccessful
diagnosticians both clinically and statistically. they indicated that successful diagnosticians had
more thorough and relevant problem representations than did the unsuccessful ones and did more
simultaneous comparing and contrasting of diagnoses. They acknowledged that differences in
knowledge or experience might explain success or unsuccessful diagnoses, however the key
finding was the differences in abilities to compare and contrast among several diagnoses
simultaneously by using relevant representations of the clinical problem. They state that in doing
so, the problem solver can construct a mental representation of the situation in a different
formats, using words, graphs, diagrams, tables, equations, and pictures. They further indicated
that using multiple formats for problem representation improved students’ problem-
representation skills and resolution. Chang et al. (1998) indicated that problem representation in
medicine could be viewed as abstraction of semantics; that is, words that “serve as verbal
representations of the problem that are useful in retrieving knowledge from memory, such as
visual representation of the pathophysiology or pathology (e.g., seeing in the mind’s eye an
inflamed joint with its various structures)” (p. S109). They indicated that abstraction of problem
representation in this way may correspond to greater diagnostic depth and accuracy, reflected in
elaborated or compiled discourses, as opposed to reduced or dispersed ones. Ironically, in many
medical schools, courses in physical diagnosis (which could promote abstract representations of
a patient’s problem) and in pathophysiology (which could promote abstract or visual
representations of disease) are typically taught concurrently but with little or no integration for
students. The findings from this study suggest that educational strategies should be used to
encourage integration of semantic and visual abstractions in problem representation to improve
diagnostic skill.
In the End-of-Life Care Distance Education Program, Knowledge Building in Knowledge Forum
provides opportunities for problem representation across various formats. Technically,
Knowledge Forum enables the translation of ideas and words to text, rendering them explicit, as
knowledge objects, in a visual format. Clinical videos have been used to provide visual context.
However, visual problem representation has not been used in the problem space, although
technically the opportunity exists. Hence, Knowledge Building through words and images in
Knowledge Building in Continuing Medical Education … 53
open, problem space in Knowledge Forum can be used not only to enable visualization of
cognitive artifacts and processes, but also to represent problems.
A follow-up study demonstrated the positive outcomes of making clinical reasoning “more
visible and conscious,” as well as “who learns from whom” (Nendaz, Junod, Vu, & Bordage,
1998, p. S54). The use of a group management plan, called a metaplan, was created for students,
residents, and house staff to use in educational rounds. A short clinical vignette (in text) from a
real patient’s record was displayed on a pin board. Each participant, was given an anonymous
code number, was asked to write three ranked diagnostic hypotheses and a justification of each
hypothesis on a separate cards. Then the entire group was asked to pin up their cards (diagnosis
and justification side-by-side) and to group them by rank, on the board. Participants perused all
displayed card and then were allowed to post revisions. This rudimentary, pushpin display
method can be seen as a low-tech version of a structured exercise that could be more easily
conducted by creating Knowledge Forum notes for the visual display of cognitive artifacts in an
online problem space.
Cards were collected and scored by five physicians, using four categories: correct and complete
final diagnoses, correct but incomplete diagnoses, plausible diagnoses/part of the differential
diagnosis, or incorrect diagnoses. Justifications were categorized as essential findings,
contributing findings and non-relevant findings; misconceptions were noted as well. In terms of
discourse, it was found that experienced physicians used more compiled language; they used a
single summary word or set or terms (e.g., “Sjogren’s syndrome”) as opposed to more elaborated
descriptions of symptoms (e.g., “dry mouth” and “conjunctivitis”) to justify diagnoses.
Most importantly for our purposes were the findings on the effects of visual displays, from this
structured, group exercise. After visual display of the initial diagnostic hypotheses, subjects, with
less than eight years of training/practice, made on average two changes, such as adding or
deleting a diagnosis, changing the order of diagnoses, or refining an existing diagnosis. Subjects
with more than eight years of training made only one change. The addition of new diagnoses
were more frequent among fourth-year students than among physicians with more than four
years of experience. “Overall, more than three quarters of the diagnostic changes were beneficial
(95% CI = 73% to 82%), reaching 92% (95% CI = 83% to 97%) among fourth-year students (x2
(4) = 12.203, p = .01)” (Nendaz et al., 1998, p. S56). The visual displays were more beneficial to
Knowledge Building in Continuing Medical Education … 54
subjects that initially listed an incomplete correct diagnosis, than to those who did not mention
the correct diagnosis initially. When a change had been made subjects were asked to note which
card had been most influential in their ideas for revision. Results indicated 70% of students relied
mainly on ideas from chief residents and residents (whose discourse has been noted to be more
elaborated than compiled in a related study by Schmidt & Boschuizen, 1992). Residents (63%)
relied mainly on cards of more experienced physicians.
Nendaz et al. (1998) noted that although 78% of diagnostic changes were beneficial in this
structured exercise, it was unclear whether students added diagnosis by simply copying ideas or
if they deeply understood the full meaning of their changes. Therefore extended opportunities for
discourse are recommended to ensure students do not simply mimic complied discourse of
experienced physicians. In addition authors note that the visual display of reasoning and
justifications also allows the visualization of misconceptions that could have remained
unnoticed. In this exercise, experienced physicians were provided with opportunities to examine
and refine their initial hypotheses. These finding have present strong evidence for the creation of
opportunities for physician elaborated discourse, as made possible in the End-of-Life Care
Distance Education Program in Knowledge Forum.
Identification of misconceptions and correction of errors have been and continue to be a key
concern in medical education and throughout practice (Elstein, Shulman, & Sprafka, 1978;
Friedman, Connell, Olthoff, Sinacore, & Bordage, 1998). However, verification of positive
benefits of visual display of knowledge and thinking process through collective work is a noted
shift in the recent literature related to identification of misconceptions, self-correction and related
notions of self-assessment and self-regulation.
Lester and Tritter (2001) argued for collaborative work: “A more theoretically informed
approach may be to address the genesis of medical thinking about error, through reforms of
aspects of medical education and socialization” ( p. 855). The authors regarded the failure of
individualistic reforms as due to the fact that medical errors are usually the product of a series of
factors, which suggests that change is need at a system rather than an individual level. Medical
knowledge often remains tacit, uncertainty is seldom articulated, and errors are almost never
discussed (Cimino, 1999; Patel, Arocha, & Kaufman, 1999). Lester and Tritter called for a shift
Knowledge Building in Continuing Medical Education … 55
in medical culture, from a “conspiracy of silence” to one of open discourse, characterized by
collegiality and cooperation.
Most recently the effectiveness of self-directed learning has been questioned (Murad, Coto-
Yglesia, Varkey, Prokop, & Murad, 2010). Concerns around individually held misconceptions
(Lester & Tritter, 2001), tacit knowledge (Cimino, 1999), and personal reflections on practice
(Charon, 2004) pose interesting possibilities for further examination for online learning (Patel,
Yoskowitz, Arocha, & Shortliffe, 2009) and collaborative Knowledge Building (Scardamalia &
Bereiter, 2003a).
We also find articulation of issues in the medical education literature around self-assessment,
self-direction, and self-regulation (Regehr & Eva, 2006), based on early work by Kruger and
Dunning (1999) who said that individuals had difficulties in recognizing their own
incompetencies, which led to inflated self-assessments. They titled their article: “Unskilled and
Unaware of It.” In a systematic review by Davis et al. (2006), physicians were shown to have a
limited ability to accurately self-assess; in fact, a number of studies found that the worst accuracy
in self-assessment was among physicians who were the least skilled and those who were the
most confident. It is noted that self-assessment was central to continuing medical education and
lifelong learning, relicensure, specialty recertification and clinical competence for continued
practice. Hence some studies suggested that instead of continuing to try to train physicians to be
more accurate self-assessors, physicians should seek feedback from peers and health care teams,
and reflect on their clinical practice. A recent Knowledge Building summary paper identified
new ways to support self-assessment and metacogntive reflection through social construction
(Scardamalia et al., in press).
Complexity of clinical discourse will be assessed in this study using the Bordage and colleagues’
(1991, 1994, 1997) rating scale and definitions.
2.4.6 Analogies and Emergent Ideas in Abductive and Adductive Processes
It is posited that sociocognitive work in the End-of-Life Care Distance Education Program,
Knowledge Forum problem space will result in authentic discourse around emergent clinical
scenarios. Evidence of real-world, practice-based cases and communally situated explanations
and ideas articulated in the Knowledge Forum problem space may provide important examples
Knowledge Building in Continuing Medical Education … 56
of the use of analogies (Glick & Holyoak, 1983; Holyoak & Thagard, 1995; Thagard, 1997) and
abductive reasoning (Thagard & Shelley, 1997), in knowledge building. Use of analogies is
advantageous for transfer of ideas to practice or what Thagard calls coherence in through and
action (Thagard, 2000, 2007).
Thagard (1997) described six uses of medical analogies that illustrate different kind of analogical
transfer: theoretical, experimental, diagnostic, therapeutic, technological, and educational
analogies. Distinction of types of analogies with the discourse is beyond the scope of this study.
However, notation of inclusion in the discourse will been seen to be indicative of abductive
reasoning.
Thagard and Shelley (1997) defined abductive reasoning as “reasoning in which explanatory
hypotheses are formed and evaluated” (p. 413). Much of knowledge building work is to create,
make sense of, and debate explanatory hypotheses or theories. Evaluation of knowledge building
discourse through examination of coherence of ideas can provide stimulus for further discourse
and opportunities for external assessment. Thagard and Shelley (1997) indicated: “Explanation is
not deduction; hypotheses are layered; abduction is sometimes creative; hypotheses may be
revolutionary; completeness is elusive; simplicity is complex; and abductive reasoning may be
visual and non-sentential” (p. 413).
Important to knowledge building discourse is Thagard’s (2007) notion of coherence. He said:
“We can judge that a scientific theory is progressively approximating the truth if it is increasing
its explanatory coherence is two key respects: broadening by explaining more phenomena, and
deepening by investigations layers of mechanisms” (p. 28).
In this study, Thagard’s (2007) concepts of broadening and deepening are used to further clarify
our understanding of knowledge building. What I call abduction in this dissertation, Thagard
calls broadening. What I call adduction is his deepening. Knowledge building work aims to
broaden and deepen ideas. Abductive and adductive knowledge work can also be considered
reflective of Bereiter’s (2000) concept of “drilling down and building up” (p. 205). These
nuances and relationships provide important clarifications and help further our understanding of
the complexities of knowledge building.
Knowledge Building in Continuing Medical Education … 57
Within-note content analyses of knowledge building discourse were conducted in this study to
examine possibilities of analogical thinking, emergence, and abductive and adductive knowledge
work.
2.5 Summary and Implications
This review of the literature has demonstrated a paucity of research studies on Knowledge
Building in the context of continuing medical education. No papers were found that use social
network analysis to describe attributes and relationships amongst dimensions that successful
scaffold and support Knowledge Building.
Recent literature in continuing medical education calls for a change. The specific change
required is to improve effectiveness (Moores, Dellert, Baumann, & Rosen, 2009); it stresses the
importance of moving beyond self-learning to a community context (Dorman & Miller, 2011;
Sargeant, Mann, van der Vleuten, & Metsemakers, 2008; University of Toronto, 2011), while
remaining evidence-based (Dauphinee & Wood-Dauphinee, 2004).
New avenues and opportunities for research and development have been identified in the
literature, such the one recently published by the University of Toronto (2011): “The Office of
CEPD is seeking to change problem solving into possibilities for breakthrough thinking. This
includes thinking about continuing education as a type of creative experiment” (p. 1). It is
anticipated that recommendations such as this will enable new opportunities for creative
community engagement in continuing medical education, which will require corresponding
measures and new ways to describe social network interactions and relationships beyond those
captured by individual participation assessments.
This study seeks to fill this gap in the literature and attempts to go beyond descriptions of
individual interactions and knowledge gains to provide an understanding of Knowledge Building
community interactions, network structures, and the relationships that can successfully scaffold
knowledge improvement in continuing medical education. A new, socially constituted model of
Knowledge Building requires elaborated, detailed descriptions of relationships, as opposed to
mere statements of summative findings (Kanter, 2008). Results of this study and the analysis of
Knowledge Building in Knowledge Forum in the End-of-Life Care Distance Education Program
Knowledge Building in Continuing Medical Education … 58
are intended to evaluate potentiality, to contribute to the forward progression of current
recommendations for improvement in continuing medical education.
The pedagogic design and research methods employed in this study will be described next.
Knowledge Building in Continuing Medical Education … 59
CHAPTER 3
METHODS
3.1 Introduction
This chapter begins with a description of the context of this study, a statement of the goal of this
research study and the research question, followed by an overview of methodology and pedagogic
design. It then focuses on the research methods, detailing study population, instruments and
procedures, and data collection and analyses employed. This chapter concludes with a summary
table of research questions, data collection, and data analyses, framed in relation to the three study
clusters.
3.2 Context and Program Overview
This study was conducted in the context of a continuing medical education course for family
physicians, called the End-of-Life Care Distance Education Program. Participants in this
program are recruited by flyer mailed to all family medicine physicians in the Toronto, York,
and Simcoe regions, by the Office of Continuing Education and Professional Development,
Faculty of Medicine University of Toronto. Registration is open annually to the first 25
participants that respond by phone, email, or regular mail. Cost of participation is covered by the
Ontario Ministry of Health and Long-Term Care. A research ethics information and consent form
was provided to all participants (Appendix A).
The program began each year in early October and ended in early May. There were five case-
based online modules and two blended learning modules (blending videoconferenced, face-to-
face session with follow-up discussion) in the End-of-Life Care Distance Education program.
Only the five case-based online modules are included in this research study. Each case-based
module was conducted in Knowledge Forum over the course of four weeks. Each group was
facilitated by a specialist in palliative care.
The number of participants in each online group with a facilitator differed in each year and
across modules, depending on number of registrants and commitment to part or the full program.
In aggregate across 2005/2009, 73 students completed all aspects of the program (see Table 2).
Four different facilitators participated across these years.
Knowledge Building in Continuing Medical Education … 60
Discourse notes written and contributed to Knowledge Forum, online pre-/posttest tests, and
attitude and opinion surveys provide the data for this research study. Data was collected across
five years, between 2004 and 2009. A mixed-methods case study methodology across years was
used.
3.3 Research Goal and Question
The goal of this thesis is to show improvements according to both traditional and non-traditional
measures—to show that there is not tradeoffs but rather improvements on both fronts. This thesis
will describe (a) traditional learning outcomes based on individual measures, (b) performance
over and above traditional measures (i.e., beyond learning as traditionally conceived and
measured), and (c) sociocognitive dynamics that enable work over and above traditional
learning.
Traditional measures of individual learning employed were online pre-/posttest outcomes and
results of attitudes and opinions surveys.
Performance measures over and above those typically concerned with traditional learning, such
as Knowledge Building measures of online activity/interactivity and social network analyses
were calculated across all groups and all years.
In-depth comparison of the sociocognitive dynamics between two groups, in the last year of this
study, 2008/2009 was also conducted, using social network analysis and content analysis. Ten
modules were analyzed to examine relationships between social network position/power.
Participants structures and ideas at the core, mid and periphery were compared with knowledge
improvement scores. Content analyses of 40% of the 2008/2009 discourse notes were performed
to gain deeper insight into facilitator/student sociocognitive dynamics, determination of
emergent themes, examination of complexity of discourse, and evaluation of evidence of
Knowledge Building indicators.
This thesis seeks to determine the following research question:
Does Knowledge Building improve physicians’ knowledge and understanding of
palliative care in a Web-based continuing medical education course, over a five-year
period; and if so, what social network structural relationships and sociocognitive
Knowledge Building in Continuing Medical Education … 61
dynamics support knowledge improvement, and contribute to democratization and
metadesign of Knowledge Building in continuing medical education?
The study was guided by this research question, and framed by the three clusters outlined in
Table 1 and found in Chapter 2 (literature review), Chapter 3 (research methods), and Chapters
4, 5, 6, and 7 (results chapters). Details on case study methodology and pedagogic design are
discussed next, followed by detailed explanation of the research methods.
3.4 Case Study Methodology
A case study approach was used in this study (Creswell, 2003, 2007, 2009; Merriam, 2001). Both
quantitative and qualitative methods were used.
Merriam states “that the single most defining characteristic of case study research lies in the
delimiting the object of study, the case” (2001, p. 27). She further clarifies by defining case study
from other research designs using Smith’s concept of a “bounded system,” Stake’s notion of an
“integrated system” (p. 27) and Cronbach’s framing of case study research as “interpretation in
context” (Merriam, 2001).
Case study research design is particularly suited to situations where it is impossible to separate
the variables from their context, as in many holistic educational research studies.
Case studies can be further defined by three special features; they are “particularistic” meaning
they focus on a particular situation; they are “descriptive” portraying as many variables as
possible and their interactions often over time; and they are “heuristic” meaning that they are
intended to illuminate understanding of the phenomena under study (Merriam, 2001, p. 29-31).
“Previously unknown relationships and variables can be expected to emerge from case studies
leading to a rethinking of the phenomenon being studied” (Stake, 1981, p. 47, as quoted in
Merriam, 2001, p. 30).
Case study research is not limited by any particular methods for data collection or data analysis.
Nor is it limited by any predefined frameworks, such as those used in program evaluation
(Posavac & Carey, 2003). Mixed-methods research focuses on collecting and analyzing both
quantitative and qualitative data in a single study to provide deeper understanding and
convergence of findings from different sources of data (Creswell, 2003).
Knowledge Building in Continuing Medical Education … 62
In the current study I employ various quantitative methods, including those typically used to
evaluate individual learning and those more recently developed to assess collective Knowledge
Building (Burtis, 2001; Teplovs, 2010; Teplovs & Scardamalia, 2007) and social network
relationships. Traditional quantitative methods are used to evaluate knowledge gains, through
pre-/posttests, and participant attitudes and opinions surveys (Dillman, 1978, 2000). The
Knowledge Forum Analytic Toolkit, is a powerful suit of analyses tools within Knowledge
Forum that provide numerical analyses and graphical representations regarding online
contribution and participation patterns (contributing notes, building on, annotating, and
referencing notes of others, etc.). Social network analyses, described in detail in the current
literature review, is used to describe relationships of participant structures, idea interactions, and
sociocognitive dynamics (Scott, 2001; Scott & Carrington, 2011; Scott, Carrington &
Wasserman, 2005).
I employ qualitative analysis on a small subset of data in this study, as Creswell (2007) indicates,
to enrich our understanding and verify relationships. I use within-note, content analyses of the
discourse in Knowledge Forum. Inductive and deductive approaches are employed (Creswell,
2007; Merriam, 2001). Discourse is analyzed deductively according to a framework of
teacher/student structural/power relationships (Mills, 1997; Hodges, Kuper, & Reeves, 2008) and
definitions of clinical complexity (Bordage, 1994), and definitions of Knowledge Building
(Scardamalia, 2002), and inductively for emergent themes. Chi’s method (1997) of quantifying
qualitative analyses is applied to the deductive analyses.
This mixed-methods multiyear case study was conducted in the unique context of the End-of-
Life Care Distance Education Program, with a study population of physicians/students and
facilitators/palliative care experts, using a series of quantitative and qualitative methods, to
describe important relationships between individual learning and collective Knowledge Building.
The research protocol for this study was approved by the Health Sciences Research Ethics Board
of the University of Toronto in 2004.
3.5 Pedagogic Design
The End-of-Life Care Distance Education Program was specifically created to fill a gap in
palliative care education for continuing medical education for physicians in practice. The
Knowledge Building in Continuing Medical Education … 63
pedagogic design of the program was based on Knowledge Building theory and the use of
Knowledge Forum for collective discourse. However, the curriculum remained rooted in a
traditional mode, framed by predefined learning objectives and a clinical case scenario.
Participants were divided into groups facilitated by an expert in palliative care. Framework for
facilitation was extended beyond that described in problem-based learning (Barrows and
Tamblyn, 1980), as a guide-on-the-side. In this course facilitators were framed as full
participants (Bereiter & Scardamalia, 1993).
3.5.1 Multimedia Design of Knowledge Forum Communal Space
A goal underlying the design of Knowledge Building environments (Scardamalia, 2003b) is to
create communities for knowledge advancement, enabled by supports for high level processes
and systems for embedded and transformative assessment within the discourse. Knowledge
Forum, is specifically design to support Knowledge Building and was used in the current study.
A website (End-of-Life Care Distance Education, 2004) was created to provide a publically
accessible information and access to the End-of-Life Care Distance Education Program. The
website contains information on the course, registration, the schedule, credits, and access to
online resources and Knowledge Forum (Appendix B).
The program opens with a videoconferenced face-to-face session, called Introduction to
Palliative Care. The next three modules are online cases. Mr. Singh’s Pain, Parts 1 and 2, focuses
on management of neuropathic pain and the case of Mary’s Misery deals with complex pain and
psychosocial issues. Next is an interactive videoconferenced, face-to-face session, exploring
palliative care symptoms other than pain (e.g., dyspnea). This session is followed by the last two
online cases, called Judy’s Last Days, Parts 1 and 2, which deal with patient and family issues
around death and dying.
Each multimedia case scenario in Knowledge Forum begins with a text-based introduction,
followed by a 3 to 5–minute clinical video vignette of the patient, followed by a prompt for
collaborative knowledge building issues for discussion (Appendix C).
Individual and collective “Reflection-on-Practice” components are integrated within each
scenario to draw out case relevance to real-world practice for emergent discussion. Digital
Knowledge Building in Continuing Medical Education … 64
references were organized by case and clinical topic, in an online library to support evidence-
based knowledge building. “Ideas as the Centre” (Scardamalia, 2003a) and related to the case
objectives, are provided to guide the collective discourse. Predetermined learning objectives are
posted for each online module. Each of the five online modules were conducted over a one-month
period of time in Knowledge Forum.
3.5.2 Knowledge Building Theory-Based Design
Knowledge Building theory-based design was the pedagogic approach employed throughout all
modules (Scardamalia and Bereiter, 2003a, 2003b). The End-of-Life Care Distance Education
Program was designed to provide a (media communal space for collaborative Knowledge
Building, with an emphasis on the Knowledge Building principles of community responsibility,
improvable ideas, embedded and transformation assessment through discourse, and
democratization of ideas (Scardamalia, 2002). The goal in the creation of a Knowledge Building
community is to engage learners in a complex, interactive process that enables them to take
charge of the educational process at the highest levels. As in the case of knowledge-creating
organizations, participants see ideas as improvable, and their goal as improving them (Nonaka,
1991). Scardamalia and Bereiter (2005) argue that complexity theory or a systems approach to
learning arises from the need to address two large problems--creativity and depth of
understanding.
Pedagogic design of the End-of-Life Care Distance Education Program can be said to reflect the
intersection of prescribed learning/assessment, as demonstrated by the predefined learning
objectives and opportunities for emergent knowledge building discourse and assessment.
Learning objectives from the three pain modules were also used to categorize subject matter
knowledge on analysis of pain pre-/posttests. The pain pretest is administered before the first
pain module, Mr. Singh’s Pain, Part 1 and the pain posttest is administered after the third pain
module, Mary’s Misery. Three other sets of pre-/posttests were administered to participants but
were excluded in this study since they were qualitative in nature.
All online pre-/posttest were developed primarily for purpose of scaffolding self-assessment, to
provide constructive feedback for individual improvement goals and collective Knowledge
Building objectives. The difficulty of accurately assessing one’s own strengths and weaknesses
Knowledge Building in Continuing Medical Education … 65
is acknowledged in the research literature (Donovan et al., 1999; Eva & Regehr, 2005; Regehr,
Hodges, Tiberius, et al., 1996). Pre- and posttests are commonly used in a summative rather than
a formative manner, to make claims about change in educational outcomes. In this study,
individual formative feedback was provided to participants online to support self-assessment,
enabling participants to identify knowledge “lacks” (Bereiter, 2002a; Scardamalia & Bereiter,
1989, 2005).
A knowledge self-assessment system, such as this, is intended to enable individual conceptual
change, as well as elevate the community Knowledge Building discourse; the ultimate goal is to
evoke deep relational understanding (Bereiter, 2002c), and to translate higher levels of
knowledge into new ideas, improved practice, and ultimately more effective patient care. Further
information on the novel pre-/posttest pedagogic design created for this program can be found
elsewhere (Lax, Singh, Librach, & Scardamalia, 2006).
The Knowledge Forum discourse environment opens opportunities for work in belief and design
modes (Bereiter & Scardamalia, 2003). The design of the discourse environment can be
characterized as open and unstructured, yet scaffolded to support Knowledge Building and
learning as a by-product of Knowledge Building. Traditionally instructional scaffolds have been
used in learning environments as temporary supports for learning that are gradually removed as
the learner achieves the intended outcome (Brown, Collins, & Duguid, 1989; Brown & Palincsar,
1989; Collins et al., 1989; Vygotsky, 1978). In a Knowledge Building environment, scaffolds are
permanent features of the environment since there is no predetermined level of achievement and
the aim is to support continual knowledge improvement (Lee, Chan, & Van Alst, 2005;
Scardamalia & Bereiter, 2003b). In the End-of-Life Care Distance Education Program,
Knowledge Building scaffolds are represented in Knowledge Forum both technically and
conceptually, within the discourse, through community responsibility, including participation of
content specialists, as well as, through learning objects and intentional efforts to raise knowledge
work beyond the predefined objectives. Authoritative resources in the online library also provide
information and evidence to scaffold Knowledge Building (Scardamalia, 2002a).
Unstructured online discourse environments have raised some concern in the literature about the
perpetuation of misconceptions (Burtis, Chan, Hewitt, Scardamalia, & Bereiter, 1993) and
possibility of misconceptions not being addressed (Hewitt, 2005). The role of the content expert
Knowledge Building in Continuing Medical Education … 66
is primarily to challenge uncaught misconceptions and if need be didactically correct them. As
well as, in the End-of-Life Care program facilitators can provide authentic, practice-based
knowledge on issues such as billing, hospital-based versus community-based practice, etc.
3.6 Research Methods
The research protocol was reviewed and approved by the University of Toronto ethics board.
This study was conducted from 2004 to 2009. Pedagogic design, including media and cases,
were developed in Knowledge Forum 2003/2004. A pilot study was conducted from October
2004 to May 2005. Feedback on the pilot study indicated the need for change to the pain
modules and pre- and posttests, to include a medication titration chart. Otherwise content
remained the same throughout this study from 2005 to 2009. Interactivity design of the
Knowledge Forum module notes were changed in 2008/2009. The lengthy scrolling module note
was changed to an interactive series of pages, like a mini-website, to improve organization.
This study is concerned with evaluation of the End-of-Life Care Distance Education Program
across five years, the 2004/2005 pilot and following four years: 2005/2006, 2006/2007,
2007/2008, 2008/2009. Across all five years, five modules in Knowledge Forum were examined:
the three pain content modules, Mr. Singh’s Pain, Part 1, Mr. Singh’s Pain Part 2, and Mary’s
Misery and the two last days modules, Judy’s Last Days, Part 1 and Judy’s Last Days, Part 2.
Online discourse in Knowledge Forum for each module was conducted over a one-month time
period.
A mixed-methods case study approach was used (Creswell, 2003, 2009; Merriam, 2001).
Measures of knowledge change, in combination with contribution and interaction measures,
social network measures, content analyses and Knowledge Building indicators, are used to
address important outcomes across years, in the absence of control-group contrasts.
Although a design research approach (Bereiter, 2002b; Brown, 1992; Collins, Joseph, &
Bielaczyc, 2004) is generally well matched to Knowledge Building research, it was not the case
in this study. Few design changes, other than in the first year of the study, were made, so design
research was not employed.
Knowledge Building in Continuing Medical Education … 67
3.6.1 Participants
Thirteen family physicians complete all aspects of the first program, pilot study. In the following
four years, 73 family physicians completed all aspects of the program and research study. In
addition four facilitators participated across eight groups over five years (Table 2).
Family physicians practicing in three different regions of southern Ontario: Simcoe, Toronto and
York, were invited by email and mailed brochure to participant in the study and program. One or
two specialists in palliative care medicine were chosen to facilitate discourse each year, based on
the number of participants registered in the program. All facilitators were in current practice as
clinicians in a large Toronto teaching hospital and were hand-selected by the content developer
of the program, Dr. Anita Singh. All facilitators were considered equally knowledgeable.
Participants were divided into groups for online collaborative work in Knowledge Forum.
Participation across the five years was as follows:
· In 2004/2005 pilot study, 18 family physicians and two facilitators participated in the
course (two groups); 16 family physicians completed the pain modules, and 13
completed the course.
· In 2005/2006, 16 family physicians and one facilitator participated in the course (one
group); 15 family physicians completed the pain modules, and 14 completed the
course
· In 2006-2007, 14 family physicians and one facilitator participated (one group); 11
family physicians completed the pain modules, and the full course.
· In 2007/2008, 31 family physicians and two facilitators participated (two groups); 26
family physicians completed the pain modules, and the full course.
· In 2008/2009, 22 family physicians and two facilitators participated; 18 family
physicians completed the pain modules, and 15 completed the course.
3.6.2 Instrumentation and Procedures
Pain pre- and posttests of knowledge, Knowledge Forum Analytic Toolkit and new tools for
measures of online discourse activity and interactivity in five modules were used, along with a
postcourse, self-reported, attitudes and opinions survey for data collection and analyses in this
study (Table 2).
Knowledge Building in Continuing Medical Education … 68
3.6.2.1 Pain knowledge pre- and posttests. Pain pre- and posttests were developed for online
use. All tests were individually password protected (but can be accessed online with
username/password=test/test). Learning objectives were categorized for the acute and complex
pain modules, and guided pre-/posttest design, development and feedback. Twenty-eight
multiple choice, fill-in-the-blank, and short answer/word recognition questions were constructed
based on the following four learning objectives:
· Understanding principles of pain management = six questions
· Use of opioids in cancer pain = seven questions
· Understanding neuropathic pain = five questions
· Understanding bone pain = 10 questions
Questions on the pain pre-/posttests were exactly the same. Tests were completed online
and automatically dated and time stamped. Upon completion of the pretest participants were
given individual Score Cards based on the four pain objectives to aid self-assessment and
identification areas of strength and weakness, particularly knowledge “lacks” and/or possible
misconceptions. Participants were asked to read references provided in the online library
(according to objectives) and collaboratively contribute to the online discourse, focusing on
necessary areas for Knowledge Building.
Upon completion of the three pain modules, after three months of online discourse, participants
were asked to complete the pain knowledge posttest. Data was collected online. Participants
were given individual comparative pre-/posttest Score Cards, the correct answers, in depth
explanations for each answer, and references to individually address any outstanding knowledge
issues around the subject of pain in palliative care.
Knowledge Building in Continuing Medical Education … 69
Table 2
End-of-Life Care Distance Education Program Participants and Data Collection Instruments
EVALUATION COMPONENTS AND NUMBER OF PARTICIPANTS
2004/2005
(Pilot)
2005/2006 2006/2007 2007/2008 2008/2009 2005–2009
Students Students Students Students Students
G1 G2 G1 G1 G1 G2 G1 G2 Total
Pain Pretest
(matched)
9 11 7 26 10 7 61/73 =
83.56%
Mr. Singh’s
Pain, P1
8 11 15 11 14 17 10 9
Mr. Singh’s
Pain, P2
8 11 15 11 15 16 10 9
Mary’s
Misery
16 15 11 14 16 10 9
Pain Posttest
(matched)
9 11 7 26 10 7 61/73 =
83.56%
Last Days P1 14 15 11 13 16 10 9
Last Days P2 13 15 11 13 15 10 9
Attitudes and
Opinions
Survey
12/13
(77%)
12/15
(80%)
10/11
(91%)
26/28
(93%)
12/19
(63%)
60/73 =
82%
Facilitators 1A 1B 1C 1C 1C 1D 1C 1D
Note. Facilitator A (palliative care physician—Simcoe) facilitated one group in the Pilot Study. Facilitator B
(palliative care physician—Sinai) facilitated one group in the Pilot Study. Facilitator C (palliative care physician-
Sunnybrook) facilitated one group for four years. Facilitator D (palliative care physician—Sunnybrook) facilitated
one group for two years (took the course first).
3.6.2.2 Attitudes and opinions survey. An online Attitudes and Opinions Survey was
constructed, using a 5-point Likert scale and Yes/No questions to examine participants’
satisfaction with their continuing education experience and collaborative knowledge building
(Dillman, 1978, 2000). The survey was linked in a Knowledge Forum note, and data was
collected online. Participants received an email invitation to complete it, at the conclusion of the
course, and a reminder email two weeks later.
Knowledge Building in Continuing Medical Education … 70
3.6.2.3 Knowledge Forum online activity and interactivity. Five (media, case-based modules
were created in Knowledge Forum for collaborative knowledge building around palliative and
end-of-life care issues. Students and facilitators worked on each module for a one-month period,
writing, building-on, and reading each other’s notes in Knowledge Forum. These notes represent
data collected in Knowledge Forum. Learning objectives were provided online to participants for
each module. These objectives were designed by course content developers, experts in palliative
care and deemed to represent the body of knowledge to be acquired in this course. Table 2
provides a summary of participants and data collection instruments used in this study.
3.6.3 Data Collection and Analyses: Pilot Study 2004/2005
Participants completed pain pre- and posttests online. Data was collected automatically,
downloaded to Excel, matched, and then analyzed in SPSS. Pre to posttest scores were analyzed
for significant difference and effect size. Attitudes and opinions on the program were collected
by online survey. Data was time and date stamped. Analysis was performed in SPSS. Online
activity and interactivity were calculated using the Knowledge Forum Analytic Toolkit for read,
write, and build-on measures.
3.6.4 Data Collection and Analyses Cluster 1: Traditional Measures Across Years
2005/2009
3.6.4.1 Analyses of pain knowledge improvement. Participants’ pre- and posttest data were
automatically collected online (date and time stamped). These data were downloaded, matched
and scored in Excel and SPSS. Some participants completed only the pretest; these unmatched
data were scored but not used. The 2004/2005 pilot study data was not included in the
cumulative analysis since changes were made after the 2004/2005 course to specific pre- and
posttest questions to validate tests for continued use. Pain Knowledge pre- and posttests matched
results were analyzed by year, for each of the four years and cumulatively, across 2005–2009.
3.6.4.2 Attitudes and opinions of collaborative online work. Online data from the Attitudes
and Opinions Survey were collected automatically, date and time stamped, and downloaded to
Excel files for descriptive analysis in SPSS. Data was analyzed by year and cumulatively.
Cumulative analysis of the 2005–2009 results are reported in the current study.
Knowledge Building in Continuing Medical Education … 71
3.6.5 Data Collection and Analyses Cluster 2: Activity, Interactivity, and Social Network
Measures Over and Above Traditional Measures
Online activity statistics were generated by the Analytic Toolkit (ATK) in Knowledge Forum.
This is a powerful set of tools that can analyze data in numerous ways. The ATK was used in this
study to analyze read/write statistics, measures of interactivity, and more complex correlations
involving social network analyses.
3.6.5.1 Knowledge Forum online activity and interactivity. All asynchronous online notes
written and contributed by participants are saved on the server and appear within Knowledge
Forum. The Knowledge Forum Analytic Toolkit was used to calculate activity measures, i.e.
number of notes created and percent of notes read, as well as, interactivity measures (i.e., percent
of notes built-ons). These measures were calculated across all groups and at the conclusion of
each of the five online modules. Participant, including facilitator, measures were performed for
program years 2005/2006, 2006/2007, 2007/2008 and 2008/2009.
3.6.5.2 Knowledge Forum graphic contribution and social network analyses. In 2008, the
Knowledge Forum ATK was replaced with a more extensive set of analytic tools, including
graphic contribution tools and social network analysis tools. These new Knowledge Forum
assessment tools were used to analyze two groups of data in the 2008/2009 End-of-Life Care
Distance Education Program. Participant read/write contributions of 2008/2009 Group 1 and
Group 2 were analyzed along with social network measures of network links and network
density of who built-on whose notes and who read whose notes.
3.6.5.3 2-way ANOVA. A 2-way ANOVA was performed, in SPSS, on pain knowledge pre-
/posttest results to determine if there was a significant difference across and between 2008/2009
Group 1 and Group 2.
3.6.5.4 Social network structural analyses. Detailed social network analyses were conducted
on the 2008/2009 Knowledge Forum Analytic Toolkit results of participant notes built-on.
Netminer 3© software was used for structural analyses on 2008/2009 Group 1 and 2 discourse
across all 10 modules. The following social network structural analyses were conducted:
· Frequency, network edges, network density, Eigenvector centrality, in-degree centrality,
out-degree centrality
Knowledge Building in Continuing Medical Education … 72
· Cohesion index, cliques
Social network analyses of 2008/2009 Groups 1 and 2 examined structural dimensions over time
and between networks (by student only and by student/facilitator) to compare and contrast
dimensions, to determine what social network relationships support knowledge improvement
and democratization of participation.
3.6.5.5 Significant difference and effect size of social network density and centrality
measures between groups, with and without facilitator. Social network measures between
2008/2009 Group 1 and Group 2 were analyzed for significant difference (t tests) and effect size
(Cohen’s d) in SPSS. These analyses were conducted with and without the facilitator for each
group, across all modules, to determine over facilitator effect.
A second analysis was conducted to determine significant difference and effect size between
Group 1 and Group 2, with facilitator, in regards to network density and centrality measures of
build-on notes and notes read. Network centrality measures in both analyses included
Eigenvector centrality, in-degree, and out-degree.
3.6.5.6 Relationship of social network structural analyses of the three pain modules to
pain pre-/posttest scores. Clique and cohesion measures were compared between 2008/2009
Group 1 and Group 2 for the three pain modules; pain pre-/posttest results were compared.
Next, tests for significant difference and effect size were conducted on measures of density and
centrality using the three pain modules. Because of the low numbers nonparametric correlation
(Spearman’s rho) was used. Groups 1 and 2 were combined for Spearman’s correlation of pain
posttest score with social network centrally variables.
3.6.6. Data Collection and Analyses Cluster 3: Social Network Analyses and Measures of
Sociocognitive Dynamics That Support Knowledge Building Outcomes, Over and Above
Learning
3.6.6.1 Social network position and power analyses. This social network analysis was
conducted based on the ATK data from the 2008/2009 Group 1 and 2 discourse across all 10
Knowledge Building in Continuing Medical Education … 73
modules. Netminer 3 software was used for centrality position and idea power analyses of build-
on notes.1
3.6.6.2 Social network content analyses.
3.6.6.2.1 Social network analysis of facilitator and student patterns of discourse. Forty
percent of the 2008/2009 dataset were analyzed. The same two modules in Group 1 and Group 2
were selected: Mr. Singh’s Pain, Part 2 and Judy’s Last Days, Part 2. Selection was made by the
researcher based on highest core performance in previous analyses of shared power.
Two analyses were conducted. The first examined patterns of facilitator discourse. Facilitator
participant structures were categorized according to a modified protocol based on Tabak and
Baumgartner (2004, Table 1, p. 403), definitions of “monitor, mentor, or partner” categories. I
added the category of expert to highlight episodes of didactic teaching. I also added
subcategories of partner/expert and mentor/partner since some facilitator notes exhibited both
stances. This analysis was scored by a second rater for interrater reliability of findings.
The second analysis in this section examined patterns of discourse statements and questions.
Unlike Zhang et al.’s (2002) study, based on advancement of Knowledge Building through
statements, this study takes the perspective that Knowledge Building is advance not only by
statements but also by questions. In this study, I ask: Who asks the questions students or the
facilitator and from what social network position? These analyses can be considered a form of
qualitative social network analysis of facilitator/student participant structures.
3.6.6.2.2 Content analysis of themes of discourse, beyond learning objectives. Forty
percent of the 2008/2009 dataset were analyzed. The same two modules in Group 1 and Group 2
were selected: Mr. Singh’s Pain, Part 2 and Judy’s Last Days, Part 2. Within-note qualitative
analysis was conducted; threads of discourse were categorized (Zhang, et.al, 2002). Themes were
identified as emergent, beyond those listed in the predetermined learning objectives, or related to
the learning objectives. Emergent ideas were reviewed as abductive and adductive knowledge
work—that is, whether they broadened or deepened ideas.
1 Note. Social network measures of notes read was consistently high across all modules and similar between
groups and therefore no further analyses were done on these; only build-on notes were used in further analyses since
they were more variable and offered the possibility to demonstrate difference and gain a deeper understanding.
Knowledge Building in Continuing Medical Education … 74
3.6.6.2.3 Complexity of clinical discourse. Group 1 and Group 2 notes in the 2008/2009
dataset were analyzed. The same two modules in each Group were scored. Within-note discourse
was analyzed using the Semantic Analysis of Clinical Discourse scale created by Dr. George
Bordage (1994). Knowledge Forum notes were scored on a 4-point scale, rating discourse as
reduced, dispersed, elaborated, or compiled. Administrative notes were placed in a separate
category. This analysis was scored by a second rater for interrater reliability of findings.
3.6.6.2.4 Evidence of Knowledge Building indicators and exemplars. Analyses of
2008/2009 Groups 1 and 2 Knowledge Forum notes were conducted. Four modules (40%) from
the 2008/2009 dataset were analyzed; the same two modules from Group 1 and the same two
modules from Group 2. Evidence of Knowledge Building indicators was scored using Sibbald’s
(2009) scoring matrix based on Scardamalia’s (2002) definitions of the 12 principles of
Knowledge Building. This analysis was scored by a second rater. Exemplars of Knowledge
Building indicators were selected from the dataset. Knowledge Building indicators are used to
establish evidence of Knowledge Building sociocognitive dynamics, such as community
responsibility, idea improvement, democratization of knowledge, epistemic agency and emergent
work demonstrating cocreation of knowledge and metadesign of curriculum in a continuing
medical education course.
3.6.7 Summary
Various data collection and data analyses methods were used in this mix methods, multiyear,
case study. Quantitative statistical analyses, social network analyses, and qualitative analyses
were conducted. Table 3 summarizes demographics of participants, and Table 4 summarizes
research questions and the associated data sources and analyses.
Knowledge Building in Continuing Medical Education … 75
Table 3
Demographics of the Study Population
DEMOGRAPHICS 2005/2009
Respondents (N=60/73) 82%
1. Gender Male Female n/a
35(58.3%) 24(40%) 1(1.7%)
2. Region of practice Toronto York Simcoe/other
29(43.8%) 13(21.7%) 15(26.7%)
3. Area of practice/specialization Family medicine Specialist n/a
56(93.3%) 3(5%) 1(1.7%)
4. Year of graduation 1958-2008
5. Years in practice 0-5 5-10 11-20 More than 20 years
9(15%) 9(15%) 15(25%) 25(41.7%)
Table 4
Research Question/Subquestions, Data Analysis, and Data Sources
RESEARCH QUESTION
Does Knowledge Building improve physicians’ knowledge and understanding of palliative care in a Web-based
continuing medical education course, over a five-year period; and if so, what social network structural
relationships and sociocognitive dynamics support knowledge improvement, and contribute to democratization
and metadesign of Knowledge Building in continuing medical education?
SUBQUESTIONS
CLUSTER 1 TRADITIONAL OUTCOME MEASURES
Subquestion 1 Did students’ pain knowledge improve from pre to posttest and if so, is this improvement
significant and what is the effect size?
Data Analysis Data Sources
2-tailed t-test Matched pre-/posttests (Pilot and four years)
Cohen’s d effect size Matched pre-/posttests (Pilot and four years)
Subquestion 2 Traditionally continuing medical education is a self-directed or didactic experience; what are
participants’ attitudes and opinions towards collaborative online knowledge building in the
End-of-Life Care Program?
Data Analysis Data Sources
Likert scale and Yes/No questions Online survey (Pilot and four years)
Knowledge Building in Continuing Medical Education … 76
Table 4 (continued)
CLUSTER 2 PERFORMANCE OVER AND ABOVE TRADITIONAL MEASURES
Subquestion 3 What are participants’ online activity and interactivity measures, by module, by group, by
year, in the 2004/2005 pilot and 2005–2009 study (in each of the five modules, across two
groups, and over four-years)?
Data Analysis Data Sources
ATK tools in Knowledge Forum Five online modules, by group each year (Pilot
and four years)
SNA tools in Knowledge Forum Ten online modules, over two groups
(2008/2009)
Subquestion 4 Is there a significant difference in 2008/2009 Groups 1 and 2 pain knowledge scores from
pretest to posttest, and is there a significant difference between groups?
Data Analysis Data Sources
2-way ANOVA 2008/2009 Groups 1 and 2 pre-/posttests
Subquestion 5 Pain knowledge improved significantly in both groups; however, knowledge increase was
greater in Group 1 than in Group 2. What are the social network structural differences
between groups that supported increased knowledge improvement and how are these
differences related to centralization and/or democratization of participation and ideas?
Structural SNA Data Analysis Data Sources
Network edges
(measures and visualizations)
2008/2009 Groups 1 and 2, five modules,
build-ons and notes read, with/without
facilitator = 20 in total
Network density
(measures and visualizations)
2008/2009 Groups 1 and 2, five modules,
build-ons and notes read, with/without
facilitator = 20 in total
Eigenvector centrality
(measures and visualizations)
2008/2009 Groups 1 and 2, five modules,
build-ons and notes read, with/without
facilitator = 20 in total
In-degree centrality
(measures and visualizations)
2008/2009 Groups 1 and 2, five modules,
build-ons and notes read, with/without
facilitator = 20 in total
Out-degree centrality
(measures and visualizations)
2008/2009 Groups 1 and 2, five modules,
build-ons and notes read, with/without
facilitator = 20 in total
Cohesion index and cliques
(measures: cohesion of ideas, no. of
cliques, no. of members, members, etc.)
2008/2009 Groups 1 and 2, five modules,
build-on notes, with facilitator
Correlation of social network centrality
measures and knowledge gains
(Spearman’s rho)
2008/2009 Groups 1 and 2, SNA of three pain
modules and pain posttest outcomes
Knowledge Building in Continuing Medical Education … 77
Table 4 (continued)
CLUSTER 3 SOCIOCOGNITIVE DYNAMICS THAT ENABLE WORK OVER AND ABOVE
TRADITIONAL LEARNING
Subquestion
6
What are the social network relationships between structural position, power (defined as
centrality of ideas), and knowledge improvement; and how are these relationships reflected in
facilitator/student sociocognitive dynamics of Knowledge Building (beyond learning objectives),
through emergent themes, in complexity of discourse, and by indicators of Knowledge Building?
Data Analyses Data Sources
a. NETWORK POSITION AND POWER
(measures and mapped visualizations)
Structural analysis of position/idea centrality/power
and relationship to knowledge improvement
2008/2009 Groups 1 and 2, five modules,
build-ons (with facilitator),
= 10 social network centrality maps
Identification of whose ideas are at the core, mid,
or periphery (facilitator and/or students; Aviv
et al., 2003)
Relationship of student network position to
individual pre-/posttest knowledge gain Pre-/posttest results by individual and
social network centrality maps
b. FACILITATOR/STUDENT SOCIOCOGNITIVE DYNAMICS
Facilitator stance
Facilitator as monitor, mentor, participant, or
expert (modified protocol by Tabak &
Baumgartner, 2004)
2008/2009 Groups 1 and 2, four modules
(40% of dataset) to be scored by PI (and
20% by second rater), within-note analysis
Who asks the questions that drive knowledge
work?
As above
c. ANALYSIS OF EMERGENT THEMES (OVER AND ABOVE
LEARNING OBJECTIVES)
Categorization of themes and threads based on
predefined learning objectives and emergent ideas
(Zhang et al., 2009)
2008/2009 Groups 1 and 2, four modules
(40% of dataset) to be scored by PI,
within-note analysis
Relationship of emergent themes to abductive and
adductive knowledge improvement
As above
d. ANALYSIS OF COMPLEXITY OF DISCOURSE
Use of semantic analysis of clinical discourse scale
(Bordage, 1994), 4-pt rating scale of clinical
discourse as reduced, dispersed, elaborated, or
compiled
2008/2009 Groups 1 and 2, four modules
(40% of dataset) to be scored by PI (and
30% by second rater)
e. ANALYSIS OF KNOWLEDGE BUILDING INDICATORS
Evidence of Knowledge Building principles,
particularly community, democracy, agency, and
improvable ideas related to Knowledge Building
intentionality and emergent ideas/themes
2008/2009 Groups 1 and 2, four modules
(40% of dataset) to be scored by PI and
second rater
Use of scoring protocol by Sibbald (2007) and
definitions by Scardamalia (2002)
78
CHAPTER 4
RESULTS—PILOT STUDY
4.1 Introduction
The End-of-Life Care (EoL Care) Distance Education program pilot study was conducted
between September 2004 and May 2005. Traditional pre-/posttest and attitude and opinion
survey measures were employed, as well as online activity and interactivity measures (that go
beyond these traditional measures) from five online discourse modules in Knowledge Forum.
Typical online discourse read/write measures were collected along with the more innovative,
collective Knowledge Building measure of build-on notes. A total of 19 students and two
facilitators participated in the first module; Group 1 was composed of eight family physicians
plus an expert in palliative care, and Group 2 was composed of 11 family physicians and an
expert in palliative care. Groups were amalgamated in the third module and 13 students
completed the course. The pilot study was guided by the following research question:
Does Knowledge Building improve physicians’ knowledge and understanding of
palliative care in a Web-based continuing medical education course, and if so, what
social network structural relationships and sociocognitive dynamics support knowledge
improvement and contribute to democratization and metadesign of Knowledge Building
in continuing medical education?
Pilot study results were used as formative feedback for iterative improvements in design and
instrument validation.
4.2 Pilot Study, 2004/2005: Results
4.2.1 Pain Knowledge, 2004/2005: Pre- and Posttest Results
Nine participants completed both the pain pre- and posttests (for the pain online modules, Mr.
Singh, Part 1; Mr. Singh, Part 2; and Mary’s Misery). Pre- and posttests were analyzed by each
of the four pain objectives as well total score (Table 5). Mean scores, standard deviations,
confidence intervals, and effect sizes were calculated in regards to the 2004/2005 pilot test study.
A paired t-test was used to determine statistical change, across four objectives and in total. The
objectives were:
79
1. Understanding principles of pain management
2. Use of opioids in cancer pain
3. Understanding neuropathic pain
4. Understanding bone pain
Table 5
Matched Results From the EoL Care Pain Pre-/Posttests, 2004/2005
2004/2005 (N = 9)
Objective Pretest Posttest
M (SD) 95% CI M (SD) 95% CI t(8) p d
1. Pain Management 0.91 (0.17) [0.79, 1.02] 0.98 (0.06) [0.94, 1.02] 1.84 0.104 0.59
2. Opioid Use 0.40 (0.15) [0.29, 0.50] 0.47 (0.14) [0.37, 0.56] 1.28 0.237 0.49
3. Neuropathic Pain 0.68 (0.26) [0.51, 0.86] 0.83 (0.19) [0.70, 0.97] 1.61 0.147 0.66
4. Bone Pain 0.60 (0.18) [0.47, 0.72] 0.81 (0.08) [0.76, 0.87] 3.15 0.014 1.53
Total 0.54 (0.14) [0.44, 0.64] 0.65 (0.10) [0.58, 0.72] 2.35 0.047 0.89
Objective 1, Pain Management, demonstrated a very high score on the pretest and a higher score
on the posttest. Although the change was not significant, this was likely due to the ceiling effect
on pretest. Mean pretest score was 91% and posttest score was 98%. Thus, there was not much
room for improvement based on what seems to be a high level of understanding or too easy
questions in this section.
Objective 2 results, the use of opioids, also failed to show statistically significant increases in
scores. However, the reason appears very different than that seen in Objective 1. For Objective 2,
pretest scores were very low, 40%, as were confidence intervals; posttest scores were only
moderately higher, 47%. Questions and answers related to Objective 2, Opioid Use, were
carefully examined. It was found that students needed a standardized chart to optimize opioid
titration. This chart was added to the online pre- and posttests as well as embedded in the online
environment for use the following year. In addition facilitators were told in the current year and
following years that this area would require attention and much focus in the discourse.
Objective 3, Neuropathic Pain, presented positive and significant change; the mean score rose
14%, from 60% on the pretest to 83% on the posttest. Similarly, results from Objective 4 on bone
pain demonstrated a significant increase in mean scores from 54% on pretest to 81% on posttest.
80
Overall the total mean score across all objectives demonstrated significant improvement of 11%
on paired t-test = 2.35, p <.05, from pretest (54%) to posttest (65%).
Cohen’s d statistic was used to measure effect size. Guidelines used to interpret the statistic
were: .20 small; .50 medium; and .80 or more, indicative of a large effect. Effect size of the
2004/2005 program was 0.89.
To establish validity of pain pre-/posttests all questions were analyzed by item to identify
problematic questions. Items were scored and McNemar tests computed to assess whether the
difference was significant. Because of the small N (9), the power of the tests was not high.
However, these results were useful to identify problematic questions, where scores were low or
where there was a decrease or negative outcome from pre- to posttest. Graphs based on this
analysis clearly demonstrate items with positive change from pre- to posttest and items with zero
or negative change from pre- to posttest (Appendix D). Problematic questions (i.e., questions 2,
18 and 11) and answers were reviewed. Wording was changed. Titration (part questions 8, 10,
and 11, were reviewed and the need for inclusion of a titration chart in future years was
identified. Item analysis would be run again in 2005/2006 for comparison purposes and to ensure
adequate changes had been made to address problematic questions identified on the 2004/2005
pre- and posttests (Appendices D and E).
4.2.2 Pilot Study, 2004/2005: Attitudes and Opinions Survey Results
Seventy-seven percent (10 out of 13) of the participants responded to the Attitudes and Opinions
Survey (Table 6). Only one respondent had taken an e-learning course previously. One hundred
percent of respondents indicated that this continuing education experience was useful and that
they acquired new knowledge. Eighty percent of respondents indicated that as a result of this
course they would change their approach to palliative care.
Notable is the fact that pre-/posttest embedded feedback was considered helpful by 100% of
respondents. In addition, 90% indicated that this type of embedded feedback helped them
determine their individual learning needs. One hundred percent of respondents indicated
collaborative knowledge building helped them self-assess their strengths, weaknesses and
knowledge gaps, and reflect on different perspectives.
81
Most respondents (80%) indicated that the digital resources were easy to access in the online
library, called My.library. However, 40% indicated that ease of access did not prompt citation of
evidence in online discourse.
Table 6
Responses to Attitudes and Opinions Survey, 2004/2005
Demographics (n = 10 of 13)
1. My practice is located in the region of 2. I have been in practice
Simcoe 3 0–5 years 3
Toronto 6 6–20 years 3
York 1 Over 20 years 4
Continuing professional development feedback Yes (n) No (n)
3. Did you find this continuing education experience useful? 10 0
4. Did you acquire new knowledge? 10 0
5. Do you think as a result of this course you will change your approach to
palliative care? 8 2
e-Learning
6. I have taken a course using e-learning (i.e. online collaborative learning)
courseware before this one. 1 9
7. Digital resources in My.library were easy to access. 8 2
8. Access to digital references prompted use of evidence-based resources in
online discussion. 6 4
9. Pre- and posttest feedback was helpful. 10 0
10. Pre- and posttest feedback helped me determine my learning needs. 9 1
Collaborative Knowledge Building Outcomes
11. Collaborative knowledge building discourse in KF helped me reflect on
different perspective presented. 10 0
12. Awareness of my own knowledge, attitudes and opinions helped me to
self-assess my strengths, weaknesses and/or knowledge gaps. 10 0
13. Collaborative knowledge building helped me to identify changes I
would like to make in my practice. 9 1
14. Individual and collaborative reflection-on-practice sections were
helpful. 10 0
Overall evaluation Equivocal Agree Strongly agree
15. I would like to see more distance education programs for
Continuing Medical Education made available. 2 2 6
16. Based on my experience of collaborative e-learning, I
would recommend it to a colleague. 1 6 3
Average
Above
average Excellent
17. Overall I would rate the online collaborative knowledge
building component of this program as: 2 6 2
18. Overall I would rate the videoconferencing component of
this program as: 1 5 4
82
Eighty percent of respondents would like to see more continuing medical education courses
online and 90% indicated they would recommend this particular course to a friend. Overall, 10 of
13 (80%) of respondents rated the Knowledge Building component of this program, on a 5-point
Likert scale, as above average or excellent. Ninety percent rated the videoconferencing
component as above average or excellent. Use of embedded pre-/posttests as formative feedback
was determined to be helpful to participants in identifying learning needs and knowledge gaps.
4.2.3 Pilot Study, 2004/2005: Online Activity Analytic Toolkit Measures
Knowledge Forum Analytic Toolkit results suggest strong levels of online read/write activity
(Table 7). Of particular significance is the percentage of notes read (74% to 85%) by participants
across all modules. Results of percent of notes linked in Group 1 does not reach over 50% in the
first two modules; in Group 2, percent of notes linked is high in Module 1 but drops to 38% in
the second modules. Percent of notes linked in combined groups is similarly low. These
measures point to the need for more attention to linking concepts and the need for greater support
for participants to encourage this type of activity. The importance of build-on concepts will be
made explicit in the introductory session to Knowledge Building and by facilitator modelling
online.
Table 7
Online Activity Measures by Group, 2004/2005
Number of notes
created
Percent of notes read
Percent of notes
linked
Modules Group 1 Group 2 Group 1 Group 2 Group 1 Group 2
Mr. Singh’s Pain, Part 1 61 30 83 75 49 72
Mr. Singh’s Pain, Part 2 35 22 74 76 46 38
Mary’s Misery 43 75 32
Judy’s Last Days, Part 1 86 85 48
Judy’s Last Days, Part 2 29 75 19
Note. Group 1 (8 students +1 facilitator) n = 9; Group 2 (11 students + 1 facilitator) n = 12. Amalgamated groups:
Mary’s Misery (16 students + 2 facilitators) n = 18; Judy Part 1 (14 students + 2 facilitators) n = 16; Judy Part 2 (13
students + 2 facilitators) n = 15.
It is recommended that the Analytic Toolkit data be used for concurrent, embedded feedback in
the future. In this study, the Analytic Toolkit was used by the researcher to determine individual
continuing professional development credits for participants, for iterative program design, and
for data analysis.
83
4.2.4 Pilot Study, 2004/2005: Summary and Iterative Design Recommendations
In summary, aggregate scores of pre-/posttests demonstrated a positive increase of 11%
knowledge gain in pain understanding, a strong effect size of .89, high levels of online
participation in Knowledge Forum, and overall program satisfaction. However, despite these
positive results, the range, particularly on opioid scores on the pain posttest, was low. It appeared
based on item analysis that the low range of scores in this area was due to confusion about
titration and lack of knowledge in that area. The issue was not one of mathematics but instead
one of agreement on titration conversion scales and as the facilitators indicated, developing an
understanding of the art as opposed to the science of use of these medications, and how each
patient reacts differently. This is a key issue and all facilitators agreed that this problem should
be addressed with the addition conversion charts embedded in the case study and online tests,
and through more focused discourse in Knowledge Forum. Problematic questions were changed
on both pre- and posttests.
Contrary to common summative evaluation purposes, pre-/posttest in this program was used
primarily for formative self-assessment and iterative design of the program. In our model each of
the 32 questions was categorized and scored by objective. At completion of the pain pretest
participants were provided with a score card sorted by objective, providing quantitative feedback
information and recommended resources for addressing knowledge gaps and/or further learning.
Upon completion of the posttest participants were given the correct answers to all questions, in
depth explanations for each answer and further references. Novel to the project, learners were
then given an individual comparative score card, comparing their pre- and posttest scores by
objective and total scores, to help them self-assess change in knowledge and identify areas
weakness to scaffold continued improvement. Further details can be found elsewhere (Lax,
Singh, Scardamalia, & Librach, 2006).
Many studies have concluded that it is difficult for individuals to accurately assess their strengths
and weaknesses, or identify their own deficiencies (Eva & Regehr, 2005; Regehr, Hodges,
Tiberius et al., 1996). An important objective of this pilot study was to address this issue through
a novel means of feedback to support more accurate self-assessment. The ideal of “lifelong
innovativeness” (Scardamalia & Bereiter, 2005) is rooted in the notion of continuous
improvement. This pilot study provides an encouraging model of collective Knowledge Building
84
and scaffolded self-assessment for continuing medical education and was refined based on
feedback received for future years.
85
CHAPTER 5
RESULTS—CLUSTER 1:
TRADITIONAL OUTCOME MEASURES
5.1 Introduction
A pilot study was used to inform this four-year research study. This study was guided by the
same, but expanded, main research question (Table 8) and an additional series of subquestions
(Table 9).
Subquestions were organized in to three clusters, related to: (a) traditional outcome measures; (b)
performance over and above traditional measures (i.e., beyond learning as traditionally
conceived and measured); and (c) questions regarding the sociocognitive dynamics that enable
work over and above traditional learning. Results of these each of these clusters are presented in
three results chapters: Chapters 5, 6, and 7.
Table 4 in Chapter 3 provides a map of related elements organized by clusters of research
subquestions and related data collection and analyses used to obtain results presented here and in
the following two chapters. Extracts from that table will be used in the introduction of all results
chapters to provide an overview of the contents of each (see Tables 8 and 9).
Table 8
Research Question
Research Question
Does Knowledge Building improve physicians’ knowledge and understanding of
palliative care in a Web-based continuing medical education course, and if so, what social
network structural relationships and sociocognitive dynamics support knowledge
improvement and contribute to democratization and metadesign of Knowledge Building in
continuing medical education?
Note. Extracted from Table 4, Chapter 3.
86
Table 9
Cluster 1: Traditional Measures
CLUSTER 1 TRADITIONAL OUTCOME MEASURES
Subquestion 1 Did students’ pain knowledge improve from pre- to posttest
and if so, is this improvement significant and what is the
effect size?
Data Analysis Data Sources
2-Tailed t-test Matched pre-/posttests
(4 years)
Cohen’s d Effect Size Matched pre-/posttests
(4 years)
Subquestion 2 Traditionally continuing medical education is a self-
directed or didactic experience; what are participants’
attitudes and opinions toward collaborative online
Knowledge Building in the End-of-Life Care Program?
Data Analysis Data Sources
Likert and Yes/No
responses
Online attitudes and
opinion survey (4 years)
Note. Extracted from Table 4, Chapter 3.
Chapter 5 presents the results of the first cluster of subquestions related to traditional outcome
measures of learning, including knowledge gains based on pre-/posttest analysis, effect size, and
feedback on attitudes and opinion of their online experience. Results for participants the initial
year of the End-of-Life Care Distance Education Program, 2004/2005 were presented separately
from the following four years, 2005–2009, since questions on the pre- and posttest were changed
during the process of validation.
5.2 Pain Knowledge Pre-/Posttest Results by Year
and Cumulatively Across Four Years
Mean pre- and posttest scores were calculated for each of the four objectives, as well as the mean
difference, standard deviation, and effect size for each program year and cumulatively across
five years: 2005/2006, 2006/2007, 2007/2008, 2008/2009. A paired t-test was used to determine
statistical change across all four pain learning objectives. Cohen’s d statistic was used to measure
effect size.
87
5.2.1 Pain Knowledge 2005/2006 Pre-/Posttest Matched Results
Eleven of 15 participants completed both pre- and posttests in 2005/2006 program. Total mean
score across all Objectives (1 to 4) demonstrated a significant improvement in pain knowledge of
8%, from 73% on pretest to 81% on posttest, on paired t-test = 3.48, p < .006. Cohen’s d
demonstrated a large effect size of .87 related to overall knowledge improvement across the three
EoL Care pain modules (Table 10).
Objective 1, Pain Management, showed high scores on pretest, 89%, and on posttest, 92%,
indicative of a 3% mean difference. There was not much room for improvement, based on a high
level of knowledge in this area and the retention of easy questions in this section of the test.
Table 10
Matched Results of Pain Pre-/Posttests, 2005/2006
2005/2006 (n = 11)
Objective Pre M (SD) 95% CI Post M (SD) 95% CI t(10) p d
1. Pain Management 0.89 (0.11) [0.82, 0.96] 0.92 (0.14) [0.84, 1.01] 0.69 .506 0.27
2. Opioid Use 0.69 (0.16) [0.59, 0.78] 0.77 (0.12) [0.70, 0.85] 2.09 .063 0.56
3. Neuropathic Pain 0.71 (0.16) [0.61, 0.81] 0.76 (0.13) [0.68, 0.84] 0.88 .400 0.33
4. Bone Pain 0.76 (0.17) [0.65, 0.86] 0.87 (0.04) [0.84, 0.90] 2.06 .067 0.65
Total 0.73 (0.09) [0.68, 0.79] 0.81 (0.07) [0.77, 0.86] 3.48 .006 0.87
Objective 2, Opioid Use, showed improvement from 68% on pretest to 77% on posttest,
demonstrating mean difference of 9%. In comparison to scores from the previous year (40% on
pretest and 47% on posttest), the 2005/2006 range on pre- and posttest improved with the
addition of online conversion charts. However, content experts (program developer and
facilitator) remained concerned about the low level of knowledge around the use of opioids as
indicated on pretest. Hence, this area was identified as a knowledge gap/lack and associated
misconceptions became a focal point for facilitated, collective, online knowledge building
discourse.
Results from Objective 3, Neuropathic Pain, presented small positive change; the mean score
rose 5%, from 71% on the pretest to 76% on the posttest, which was considered low by context
experts. Pretest results were noted for further knowledge building in the collective online
discourse.
88
Results from Objective 4, Bone Pain, demonstrated a positive increase of 11% in mean score,
from 76% on the pretest to 87% on the posttest. Analysis at the question/answer level indicated
that change was related to use of terminology for identification of physiological concepts around
bone pain.
Results of item analysis were employed to ascertain effect of questions changed from the pilot
study. Items were scored and McNemar tests computed to assess whether the difference in
proportion correct was significant. The one-tailed tests should be interpreted with the usual .05
level of significance. Items with positive pre-/posttest change and no or negative pre-/posttest
change are included in Appendix E. Questions in relation to Objective 2, use of opioids, in
particular were reviewed to determine if these questions remained problematic even with the
inclusion of a titration chart in the 2005/2006 program. Specifically the opioid (part question
10_3 remained problematic and was targeted for revision. Other questions identified for review
and revision were pain management, question 17; opioid questions, 7, 20, and 33_4; neuropathic
pain questions 32_3 and 23_2 as demonstrated on the graph indicating negative difference from
pre- to posttest. These questions were targeted for revision for the 2006/2007 program.
5.2.2 Pain Knowledge 2006/2007 Pre-/Posttest Matched Results
Seven of 11 participants completed both tests in 2006/2007 (Table 11). Total mean score across
objectives 1 to 4 on pretest was 68% and on posttest was 78%. Improvement from pre- to
posttest was 10%.
Table 11
Matched Results of Pain Pre-/Posttests, 2006/2007
2006/2007 (n = 7)
Objective Pre M (SD) 95% CI Post M (SD) 95% CI t(6) p d
1. Pain Management 0.88 (0.13) [0.78, 0.98] 0.95 (0.08) [0.89, 1.02] 1.44 .200 0.57
2. Opioid Use 0.63 (0.17) [0.50 ,0.76] 0.73 (0.12) [0.63, 0.82] 1.46 .195 0.60
3. Neuropathic Pain 0.66 (0.17) [0.53, 0.79] 0.70 (0.14) [0.60, 0.81] 0.71 .504 0.24
4. Bone Pain 0.70 (0.15) [0.58, 0.82] 0.86 (0.11) [0.77, 0.94] 2.71 .035 1.04
Total 0.68 (0.11) [0.60, 0.77] 0.78 (0.07) [0.73, 0.84] 2.15 .075 0.93
However, results on t-test were not significant, most likely due to small sample size. Analysis of
effect size demonstrated a large effect of 0.93 related to the three pain modules.
89
5.2.3 Pain Knowledge 2007/2008 Pre-/Posttest Matched Results
Twenty-six participants completed both the pain pre- and posttests in 2007/2008. Total mean
score across all Objectives (1 to 4) demonstrated a significant improvement in pain knowledge of
7%, from 68% on pretest to 75% on posttest, and on paired t-test = 2.22, p < .04. Cohen’s d
indicated a medium effect size of .64 related to overall knowledge improvement across the three
EoL Care pain modules (Table 12).
Table 12
Matched Results of the 2007/2008 Pain Pre-/Posttests
2007/2008 (n = 26)
Objective Pre M (SD) 95%CI Post M (SD) 95% CI t(25) p d
1. Pain Management 0.85 (0.20) [0.77, 0.93] 0.89 (0.19) [0.81, 0.97] 0.88 .387 0.20
2. Opioid Use 0.66 (0.15) [0.59, 0.72] 0.72 (0.19) [0.65, 0.80] 1.80 .085 0.44
3. Neuropathic Pain 0.60 (0.18) [0.53, 0.68] 0.67 (0.17) [0.60, 0.74] 1.47 .155 0.38
4. Bone Pain 0.66 (0.20) [0.58, 0.74] 0.77 (0.25) [0.67, 0.88] 2.37 .026 0.56
Total 0.68 (0.11) [0.63, 0.72] 0.75 (0.15) [0.69, 0.81] 2.22 .036 0.64
5.2.4 Pain Knowledge 2008/2009 Pre-/Posttest Matched Results
Seventeen participants completed both the pain pre- and posttests in 2008/2009. Total mean
score across all Objectives (1 to 4) demonstrated a significant improvement in pain knowledge of
14%, from 67% on pretest to 81% on posttest, and on paired t-test = 4.30, p < .001. Cohen’s d
indicated a very large effect size of 1.15 related to overall knowledge improvement across the
three EoL Care pain modules (Table 13).
Table 13
Matched Results of the 2008/2009 Pain Pre-/Posttests
2008/2009 (n = 17)
Objective Pre M (SD) 95% CI Post M (SD) 95% CI t (16) p d
1. Pain Management 0.78 (0.20) [0.68, 0.89] 0.80 (0.19) [0.71, 0.90] 0.49 .632 0.10
2. Opioid Use 0.63 (0.16) [0.54, 0.71] 0.79 (0.15) [0.72, 0.87] 3.17 .006 1.00
3. Neuropathic Pain 0.69 (0.13) [0.63, 0.76] 0.77 (0.08) [0.73, 0.81] 2.28 .037 0.61
4. Bone Pain 0.69 (0.18) [0.59, 0.78] 0.87 (0.08) [0.83, 0.91] 4.10 .001 1.42
Total 0.67 (0.12) [0.61, 0.73] 0.81 (0.10) [0.76, 0.86] 4.30 .001 1.15
90
5.2.5 Cumulative Matched Results of 2005–2009 Pain Knowledge Pre-/Posttests
Sixty-one participants completed both pain pre- and posttests across the four years analyzed,
2005 to 2009. Cumulative total mean score across all Objectives (1 to 4) demonstrated a
significant improvement in pain knowledge of 9%, from 69% on pretest to 78% on posttest, and
on paired t-test = 5.34, p < .001. Cohen’s d indicated an overall large effect size of .82 across all
years (Table 14).
Table 14
Cumulative Matched Results of the 2005–2009 Pain Pre-/Posttests
2005–2009 (N = 61)
Objective Pre M (SD) 95% CI Post M (SD) 95% CI t (60) p d
1. Pain Management 0.84 (0.18) [0.80, 0.89] 0.88 (0.17) [0.84, 0.93] 1.52 .135 0.20
2. Opioid Use 0.65 (0.16) [0.61, 0.69] 0.75 (0.16) [0.71, 0.79] 4.23 .001 0.65
3. Neuropathic Pain 0.65 (0.17) [0.61, 0.70] 0.72 (0.14) [0.69, 0.77] 2.61 .012 0.43
4. Bone Pain 0.69 (0.18) [0.64, 0.74] 0.83 (0.18) [0.78, 0.87] 5.22 .001 0.76
Total 0.69 (0.11) [0.66, 0.72] 0.78 (0.12) [0.75, 0.81] 5.34 .001 0.82
As demonstrated by year and across all years, participants’ knowledge improved in areas
important to the practice of palliative care, including pain management, opioid use, neuropathic
pain, and bone pain. Some areas such as opioid use were seen to achieve the largest gains.
Understanding concepts about bone pain improved along with the vocabulary used, as
demonstrated from pre- to posttest. Concepts in pain management showed little improve due to
ceiling effect. Questions around these concepts were maintained to highlight the importance of
this area and reaffirm previously acquired knowledge.
5.2.6 Summary of Results of Pain Knowledge Pre-/Posttests
The pain knowledge pretest was conducted online before the course and the posttest was
conducted after three months of student participation in three online cases/modules. Analysis of
matched pre-/posttests were used to determine the results of our first research subquestion: Did
students’ pain knowledge improve from pre- to posttest and if so, is this improvement significant
and what is the effect size? In answer to this question, the analyses herein clearly demonstrate
total significant improvement from pretest to posttest across all years, with variability between
objectives. The 2008/2009 program demonstrated a particularly strong effect size of 1.15 while
the 2007/2008 program demonstrated the weakest effect size (0.64). Although incoming pretest
91
participant scores were similar (67% and 68% respectively) that was a substantial difference in
posttest scores (75% in 2007/2008, and 81% in 2008/2009), explaining difference in effect size.
Despite this range cumulative effective size across all years was 0.82; according to the definition
of Cohen’s d, anything over 0.80 is considered large.
5.3 Attitude and Opinion Survey Results 2005–2009
Participant attitudes and opinions on collaborative knowledge building and program design were
collected online at the end of the program. Demographic, Yes/No, Likert scale, and open-ended
questions were used in this survey. Data were collected all four years. Since attitudes and
opinions across years were similar, a summary of results from 2005–2009 programs is provided
here.
Exceptionally positive results were reflected on all aspects of content, collaborative KB,
relevance, and overall program rating. Sixty of 73 (82%) participants responded to the Attitudes
and Opinions Survey over four years, 2005–2009 (Table 15). The cumulative response rate was
82%. Of the 73, 56 were family medicine practitioners; three were specialists; and one did not
respond; 35 indicated they were male, 24 were female, and one person did not respond. Sixty
percent indicated they had been in practice more than 16 years. Forty-four percent were in
practice in Toronto, 22% in York region, 27% in Simcoe county, and the remainder were from
other areas in southern Ontario.
Table 15
Percentaged Results of Attitude and Opinions Summative Survey, 2005–2009
Demographics %
1. Location Toronto 43.8
York 21.7
Simcoe 26.7
Other 7.8
2. Gender Male 58.3
Female 40.0
n/a 1.6
3. Specialization Family medicine 93.3
Other/missing 6.7
(table continues)
92
Table 15 (continued)
Perceptions (%)
Strongly
disagree
Disagree
Equivocal
Agree
Strongly
agree
4. The goals and objectives were clearly
stated. 0 3.3 1.7 26.7 68.3
5. The goals and objectives were met. 0 1.7 3.3 36.7 58.3
6. The program is relevant to my practice. 1.7 3.3 8.3 28.3 58.3
7. My practice behaviour will change as a
result of this program. 0 5.0 8.3 18.3 68.3
8. The program was well organized. 1.7 0 3.3 21.7 73.3
9. Sufficient time was allotted for each
module. 0 1.7 6.7 13.3 78.3
10. The program was able to cover the
breadth of issues in palliative and end-of-
life care.
0 1.7 0 31.7 66.7
11. Collaborative e-learning was an enjoyable
way of learning. 5.0 3.3 8.3 31.7 51.7
12. The program satisfied my expectations. 0 1.7 11.7 35.0 51.7
Evaluation of objectives
Yes No n/a
13. The personal, social, cultural attitudes towards death and dying. 98.3 1.7
14. The current definition and the basic principles of palliative care. 100 0
15. The ethical issues confronting dying patients, their families and their
physicians including end of life decision-making, advanced
directives, euthanasia and assisted suicide.
95.0 5.0
16. The physical, psychological, social and spiritual issues of dying
patients and their families. 100 0
17. A systematic approach to working with families of dying patients. 98.3 1.7
18. Effective pain management. 100 0
19. The management of other physical symptoms (e.g. dyspnea,
constipation, skin care, mouth care, nausea and vomiting). 100 0
20. The identification and management of psychological issues for
patients. 98.3 1.7
21. The skills in providing psychological support and educational
counseling to dying patients and their families. 93.3 6.7
22. The roles of other disciplines in providing palliative care. 93.3 6.7
23. The community resources available to support patients in their
homes. 93.3 6.7
24. The approach to the last hours of caring in the home. 100 0
25. Incorporated evidence decision-making in caring for dying patients
and their families. 95.0 3.3 1.7
Continuing education feedback
26. Did you find this continuing education experience useful? 98.3 1.7
27. Did you acquire new knowledge? 98.3 1.7
28. Do you think as a result of this course you will change your approach
to palliative care? 86.7 13.3
(table continues)
93
Table 15 (continued)
Yes No n/a
e-Learning
29. I have taken a course using e-learning (i.e. online collaborative
learning) courseware before this one. 6.7 93.3
30. My Knowledge Forum (KF) group size was just right. 93.3 6.7
31. I found contributing discourse notes in KF quick to learn and easy to
use. 91.7 8.3
32. Digital resources in My.library were easy to access. 86.7 13.0
33. Digital resources in my library were helpful. 98.3 1.7
34. Test Yourself questions were helpful. 95.0 5.0
35. Pre- and posttest feedback was helpful. 98.3 1.7
36. Pre- and posttest feedback helped me determine my learning needs. 95.0 5.0
Collaborative Knowledge Building outcomes
37. Collaborative knowledge building discourse in KF helped me reflect
on new knowledge I gained. 90.0 6.7
38. Collaborative knowledge building discourse in KF helped me reflect
on different perspective presented. 95.0 5.0
39. Collaborative knowledge building helped me to become more aware
of my colleagues’ knowledge, attitudes and opinions. 98.3 1.7
40. Collaborative knowledge building helped me to identify changes I
would like to make in my practice. 88.3 11.7
Individual Reflection and Reflection-on-Practice sections were helpful. 85.0 10.0 5.0
41. Clinical summary notes by the Group Reporters were helpful. 91.7 6.7 1.7
42. Summative notes by the Discourse Analyst were helpful. 95.0 5.0
43. Facilitator feedback during discussion was helpful. 98.3 1.7
44. Facilitator feedback on final clinical summary note was helpful. 98.3 1.7
Perceived barriers and advantages
Strongly
disagree
Disagree
Equivocal
Agree
Strongly
agree
45. A barrier to e-learning in KF is the amount of
time required to read, contribute and respond in
writing.
13.3 25.0 21.7 30.0 10.0
46. Lack of convenient computer and Internet access
is a barrier to my e-learning.
50.0 18.3 6.7 16.7 8.3
47. An advantage of e-learning is that the group
work can be done asynchronously, at any time
and from anywhere, with a computer/ Internet
connection.
3.3 0.0 1.7 21.7 73.3
48. The face-to-face sessions (videoconferences)
were an important complement to e-learning.
3.3 1.7 15.0 28.3 51.7
(table continues)
94
Table 15 (concluded)
Overall evaluation
49. I would like to see more distance education
programs for Continuing Medical Education
made available.
3.3 0.0 5.0 35.0 56.7
50. I would like to see more distance education
programs for interprofessional Continuing
Education made available.
3.3 3.3 13.3 26.7 53.3
51. Based on my experience of collaborative e-
learning, I would recommend it to a colleague.
5.0 1.7 6.7 25.0 61.7
Poor
Below
average
Average
Above
average
Excellent
52. Overall, I would rate the online collaborative
knowledge building component of this program
as:
0.0 6.7 6.7 26.7 60.0
53. Overall, I would rate the face-to-face sessions
(video-conferences) of this program as:
1.7 1.7 15.0 36.7 45.0
Note. n = 60 of 73 respondents (82%).
Four respondents had taken an e-Learning course previously; 56 (93.3%) respondents had not.
Despite this, most (91.7%) participants found contributing discourse notes in Knowledge Forum
quick to learn and easy to use.
In response to a series of Yes/No questions the following attitudes and opinions were found.
Almost all (98.3%) physician participants indicated their CME experience was useful; they
gained new knowledge (98.3%) and as a result, they would make changes in their practice
(86.7%). Most (95%) respondents indicated collaborative KB helped them reflect on different
perspectives and understand their colleagues’ perspectives (95%). Ninety percent of respondents
indicated that collaborative knowledge building discourse in Knowledge Forum helped them to
reflect on new knowledge. All most all respondents (98.3%) found facilitator feedback during
discussion helpful. Pre-/posttest embedded feedback was considered helpful by 95% of
respondents, who also indicated that this type of structured, embedded, feedback assisted them
determine their learning needs.
A 5-point Likert scale was used to more discriminately determine participants’ attitudes and
opinions on online knowledge building. Barriers to online learning are perceived by less than
half of the participants. Forty percent agreed/strongly agreed that the amount of time to read and
contribute/write online notes was a barrier. Very few (15%) agreed/strongly agreed that
computer and Internet access was a barrier. Most considered (95% agree/strongly agree) that the
95
advantage of online learning was that group work can be done asynchronously. Ninety-two
percent would like to see more distance education program for CME and 86.7% would
recommend online collaborative learning to a colleague, based on their experience in this course.
The two videoconferencing sessions (the opening Introduction to Palliative Care session and the
Symptoms Other Than Pain session) were highly rated; 81.7% of respondents rated these
sessions as above average/excellent. The online collaborative knowledge building component
was rated above average by 26.7% of respondents and excellent by 60% of respondents, totaling
86.7%.
In conclusion, although the amount of time required to read, write and contribute to online
collaborative discourse may be considered a barrier, the overall strong results provide an
overwhelming picture of satisfaction with collaborative knowledge building. Over 91% indicated
they would like to see more continuing medical education programs made available online.
In summary, survey results indicated that online collaborative knowledge building helped
participants identify their strengths, weaknesses and knowledge gaps for further cognitive work.
Reflection on different perspectives within the discourse and facilitator feedback scaffolded
knowledge building. The collective discourse was highly rated by participants and can be seen as
a form of unstructured, formative feedback that is learner-driven and emergent. The use of
pre-/posttests was also determined to be helpful to participants in identifying personal areas of
strength and weakness, and knowledge lacks. The embedded pre-/posttests created a system of
structured, formative feedback, based on standard clinician/expert knowledge expectations of
what one should know to competently practice in the field. Thus both structured and unstructured
embedded, concurrent assessments were perceived positively by participants.
Aggregate scores of the 2005–2009 pre-/posttests demonstrated strong knowledge gains in pain
understanding. These strong results appear to be related to overall positive attitudes and opinions
toward collaborative knowledge building and embedded formative feedback, as well as to
findings of high levels of activity and interactivity in online discourse, the results of which will
be presented next.
96
5.4 Summary of Traditional Outcome Measures: 2005–2009 Results of Pain Knowledge
Pre-/Posttest Results and Attitude and Opinion Survey Results
Both the pain knowledge pre-/posttest scores and the participant Attitudes and Opinions Survey
feedback demonstrated strong results (in answer to our second research subquestion). These
positive traditional individual aggregate outcome measures can be used to validate the
educational “success” of this program. However, establishing that the program “succeeded” by
improving individual knowledge and satisfaction, is very different from understanding how
individuals participated and interacted to improve knowledge within a social-mediated
environment, and if knowledge work in design mode emerged.
In the next chapter, Knowledge Forum participation and interaction patterns will be analyzed to
provide us with a window into this aspect. Results of various social network analyses will also be
presented to describe individual and collective performance over and above traditional learning
outcomes.
97
CHAPTER 6
RESULTS—CLUSTER 2:
PERFORMANCE OVER AND ABOVE TRADITIONAL MEASURES
6.1 Introduction
Chapter 6 presents the results of the second cluster of research subquestions, numbers 3, 4, and 5
(see Table 16), which describe performance over and above traditional learning measures. As
indicated previously, subquestions are organized in to three clusters related to: (a) traditional
outcome measures, (b) performance “over and above” traditional measures (beyond learning as
traditionally conceived and measured), and (c) questions regarding the sociocognitive dynamics
that enable work over and above traditional learning.
Performance “over and above” traditional learning measures based on results of analyses
performed using Knowledge Forum’s suite of analytic tools and Netminer®
software for social
network analysis are presented in this chapter.
Knowledge Forum Analytic Toolkit results of read/write activity, build-on interactivity, across
all years are shown. The final year, 2008/2009, that demonstrated largest knowledge gains was
selected for further in-depth analyses; graphic contribution and social network analysis tools in
Knowledge Forum’s Analytic Toolkit were used to determine read and build-on network
characteristics. Additional results of numerous social network structural Analyses are presented
herein, as well as analyses of network cohesion and cliques.
Table 16
Cluster 2: Beyond Traditional Measures
CLUSTER 2 PERFORMANCE OVER AND ABOVE TRADITIONAL MEASURES
Subquestion 3 What are participants’ online activity and interactivity measures, by module, by group,
and by year, in the 2005–2009 study?
Data Analysis Data Sources
ATK tools in Knowledge Forum 5 online modules each year (Pilot
Study and 4 years)
SNA tools in Knowledge Forum 5 online modules each year
(2008/2009)
(table continues)
98
Table 16 (continued)
Subquestion 4 Is there a significant difference in 2008/2009 Groups 1 and 2 pain knowledge scores from
pre- to posttest and is there a significant difference between groups?
Data analysis Data sources
2-Way ANOVA 2008/2009 Groups 1 and 2 pre-
/posttests
Subquestion 5 Pain knowledge improved significantly in both groups; however, knowledge increase was
greater in Group 1 than in Group 2. What are the social network structural differences
between groups that supported increased knowledge improvement and how are these
differences related to centralization and/or democratization of participation and ideas?
Data analysis Data sources
Structural SNA Network edges (measures and visualizations) 2008/09 Groups 1 and 2; 10 modules
in total; Build-ons and notes read
(with/without facilitator); = 20 total
Network density (measures and visualizations) as above
Eigenvector centrality (measures and
visualizations)
as above
In-degree centrality (measures and visualizations) as above
Out-degree centrality (measures and
visualizations)
as above
Cohesion index (measures and visualizations) 2008/2009 Groups 1 and 2; 10
modules in total; Build-on notes,
with facilitator
Cliques (measures: # of cliques; # of members;
members; maps)
as above
Relationship of
SNA to Knowledge
Improvement
t-test for significant difference and effect size
(comparison of SNA of build-ons between groups,
across 3 pain modules)
2008/2009 Groups 1 and 2; 3 pain
modules; social network measures of
build-on notes
Spearman correlation (of posttest scores with
social network variables)
2008/2009 Groups 1 and 2; pain
posttest scores; social network
measures of build-on notes
Note. Extracted from Table 4, Chapter 3.
6.2 Online Activity and Interactivity Measures Results
Knowledge Forum Analytic Toolkit (ATK) measures of individual and collective read and write
activity were calculated, as well as online interactivity build-on measures, by year for all four
years of the program, across all five online modules: Mr. Singh’s Pain, Parts 1 and 2, on basic
pain management; Mary’s Misery on complex pain management; and Judy’s Last Days, Parts 1
and 2, dealing with the last days of life. Participants worked in each of the five modules for one
month each. Individual and collective results were posted online for participant formative
feedback and self-monitoring purposes. These statistics were also used to provide individual
activity “report cards” for Mainpro continuing education credits at the end of the course.
99
6.2.1 Results of 2005–2009 Read, Write, Build-On Measures
Knowledge Forum ATK results demonstrated particularly strong levels of online read/write
activity within all five modules and across four years, in the 2005/2006, 2006/2007, 2007/2008
(Table 17) and 2008/2009 programs (Table 18). ATK measures indicated:
· in 2005/2006 participants read a mean of 86.3% of notes across all modules and
wrote an average of 18.67 notes per module;
· in 2006/2007 participants read a mean of 92.3% of notes across all modules and
wrote an average of 33.55 notes per module;
· in 2007/2008 participants in Group 1 read a mean of 92.6% of notes and wrote an
average of 30.93 notes per module; participants in Group 2 read a mean of 92.4% of
notes across all modules and wrote an average of 18.25 notes per module;
· in 2008/2009 participants in Group 1 read a mean of 79.33% of notes and wrote an
average of 36.82 notes per module; while participants in Group 2 read a mean of
85.7% of notes across all modules and wrote an average of 25.1 notes per module.
In 2005/2006 participants created a total of 280 notes. In 2006/2007 participants contributed a
total of 369 notes in the same modules. In 2007/2008 participants were split into two groups. The
2007/2008 Group 1 participants contributed a total of 433 notes while Group 2 participants
contributed a total of 292 notes. In 2008/2009 the participants in Group 1 contributed 405 notes
and participants in Group 2 contributed a total of 251 notes. Since reading measures were
relatively high throughout, measures of note creation were deemed to be a focal point for further
analysis.
100
Table 17
Online Activity Measures from 2005/2006, 2006/2007, and 2007/2008
Number of notes
created Percentage of notes read
Online activity 2005/2006 (n = 15)
Mr. Singh’s Pain, Part 1 72 85
Mr. Singh’s Pain, Part 2 52 88
Mary’s Misery 48 86
Judy’s Last Days, Part 1 60 86
Judy’s Last Days, Part 2 48 85
Total 280
Average/participant 18.7
M (SD) 86.0 (1.2)
Online activity 2006/2007 (n = 11)
Mr. Singh’s Pain, Part 1 90 93
Mr. Singh’s Pain, Part 2 83 88
Mary’s Misery 53 96
Judy’s Last Days, Part 1 73 95
Judy’s Last Days, Part 2 70 96
Total 369
Average/participant 33.6
M (SD) 93.6 (3.4)
Online activity 2007/2008
Number of notes created Percentage of notes read
Group 1 Group 2 Group 1 Group 2 Group 1(n) Group 2(n)
Mr. Singh’s Pain, Part 1 109 90 94.2 86.2 14 19
Mr. Singh’s Pain, Part 2 124 44 90.6 95.1 15 16
Mary’s Misery 88 82 93.0 95.8 15 16
Judy’s Last Days, Part 1 44 43 97.5 95.4 13 16
Judy’s Last Days, Part 2 68 33 97.4 92.5 13 15
Total 433 292
Average/participant 30.93 18.25
M
(SD)
94.5
(3.0)
93.0
(4.0)
101
Table 18
Summary of Online Activity and Interactivity Measures, 2008/2009
Number of notes
created
Percentage of notes
read
Number of build-on
notes
Group 1 Group 2 Group 1 Group 2 Group 1 Group 2
Mr. Singh’s Pain, Part 1 96 69 79.4 92.5 80 58
Mr. Singh’s Pain, Part 2 101 60 78.6 91.0 77 48
Mary’s Misery 78 55 86.9 92.8 60 43
Judy’s Last Days, Part 1 65 36 78.2 74.1 47 28
Judy’s Last Days, Part 2 65 31 73.6 78.1 45 20
Total number 405 251 309 197
Average/participant 36.8 25.1 28.1 19.7
M 79.3 85.7
(SD) (4.8) (8.9)
Note. Group 1 (n = 11); Group 2 (n = 10)
Knowledge Building measure of number of build-on notes were not recorded for all years (other
than 2008/2009) since the number of continuing medical education Mainpro credits awarded by
The College of Family Physicians of Canada was only based on read/write participation statistics
in this program. Other Knowledge Building measures were available in the ATK (Burtis, 2001)
such as “who reads whose notes.” These statistics are interesting and informative but were not
used in this study. The ATK results table of “who reads whose notes” was particularly difficult
to interpret since results was presented numerically cross-referenced by individual to individual.
Network change over time was almost impossible to apprehend through numerical
representations of data. This issue was addressed by the recent creation of a number of
Knowledge Building graphic tools in Knowledge Forum, that were employed to obtain in-depth
results of the 2008/2009 online activity and interactivity and relationships across dimensions,
presented in the following sections.
In conclusion, ATK results indicated a high level of activity each year, in regards, to number of
notes created and percentage of notes read online. Online activity was almost always strongest in
the first two modules on pain. Understanding opioid use was the key objective of modules 1 and
2 and understanding neuropathic pain was key to modules 1, 2, and 3. Activity was particularly
strong in 2008/2009. Closer examination of 2008/2009 groups indicated that the total number of
notes created by Group 1 (n = 11) was substantially higher than Group 2 (n = 10). In 2008/2009
102
Group 1 created 154 more notes than Group 2; on average, more than 10 notes per participant. It
is particularly important to note that Group 1 created 112 more build-on notes than Group 2.
Percent of notes read was high across both groups. Further investigation of the 2008/2009 dataset
will be done since this was the highest performing year. Relationships across dimensions and
between groups will be examined.
6.3 Results of 2008/2009 Online Performance and Social
Network Relationship Measures (Beyond Learning)
The End-of-Life Care Distance Education Program was transferred to a updated version of
Knowledge Forum that provided access to a new graphical ATK in 2008/2009. It was intended
not only to support research, but to provide participants with immediate access to up-to-date
assessments for formative feedback. Concurrent and embedded assessments were represented
graphically and statistically, as opposed to previous ATK numeric only measures. The goal was
to turn assessment over to participants, for structured feedback on their current and past
performance, fostering reflection, scaffolding self-assessment, and supporting higher-level
metacognitive work aimed at improving individual and collaborative knowledge building. By
providing participants access to assessment, it no longer is in the sole purvey of facilitators,
teachers, and/or researchers, and can be considered a step towards democratization of the
pedagogic environment. The updated ATK includes various contribution/activity measures,
social network analysis, semantic overlap, and other writing and language assessment tools
(Teplovs, 2008).
Online contribution and social activity network measures were used in this study to analyze the
Knowledge Forum discourse/design space of the 2008/2009 EoL Care Program. The graphic
contribution tool was helpful in performing comparative analyses between facilitator and student
contributions, specifically on measures of total note written, number of group notes read, and
number of build-on notes. These graphic measures were converted back to numeric measures for
consistency of representation and ease of comparison as indicated in Table 16 and included
below to present facilitator/student comparative results. In addition, the results of two social
network interactivity measures are presented below. They are “who built on whose notes” and
“who read whose notes.” Patterns of activity and interactivity are compared. These new tools
provided new types of information for analysis and understanding these social network
relationships for Knowledge Building.
103
6.3.1 Group 1, 2008/2009 ATK and Social Network Assessment Measures
Social network analysis results of each KF module, that took place over a one-month time
period, across two groups of participants (each moderated by one palliative care
expert/facilitator) are presented below. Five online modules—Mr. Singh’s Pain, Parts 1 and 2, on
basic pain management; Mary’s Misery, on complex pain management; and Judy’s Last Days,
Parts 1 and 2, dealing with the last days of life—were analyzed. Note contributions, build-ons,
and reading measures are presented graphically by participant that is particularly helpful for
student-to-student comparatives and students to facilitator comparatives. Results of social
network analysis of who built on whose notes and who read whose notes are presented in terms
of network edge measures and distribution patterns.
6.3.1.1 Group 1, Mr. Singh’s Pain, Part 1. Contribution results of analysis of the first pain
module for Group 1 demonstrated a high level of activity by one facilitator and nine participants
(Figures 1, 2, 4). Although the facilitator contributed approximately twice as many notes as other
participants, the number of notes read was more evenly distributed across all participants.
Social network analysis of build-on activity reflects good distribution (Figure 3). Social network
analysis of who read whose notes represents strongly distributed activity; network edges
measured 110 (Figure 5).
104
Figure 1. Total note contributions Group 1, Mr. Singh’s Pain, Part 1, 2008/2009
Figure 2. Total number of build-on notes created Group 1, Mr. Singh’s Pain, Part 1, 2008/2009
105
Figure 3. Social network analysis of who built-on whose notes, Group 1, Mr. Singh’s Pain, Part
1, 2008/2009
Figure 4. Total number of notes read, Group 1, Mr. Singh’s Pain, Part 1, 2008/2009
106
Figure 5. Social network analysis of who read whose notes, Group 1, Mr. Singh’s Pain, Part 1,
2008/2009
6.3.1.2 Group 1, Mr. Singh’s Pain, Part 2. Results of discourse space analysis of the second
pain module for Group 1 contributions indicated 101 notes were created, over a one-month
period, by one facilitator and nine participants (Figure 6). Overall contributions, build-ons, and
reading of notes is strong (Figures 6, 7, 9). Facilitator activity is at least double that of most
participants, but relatively lower in comparison to previous module. Participant activity is more
evenly distributed than in the first module. Participants build-on each other’s notes, as well as the
facilitators notes.
Social network analysis of participant build-on activity reflects similar distribution to that
represented graphically (Figure 8). Again, group read activity was extremely high (Figure 9).
Social network analysis of who read whose notes represents distributed activity, with similar
network edges measuring 110 (Figure 10).
107
Figure 6. Total note contributions, Group 1, Mr. Singh’s Pain, Part 2, 2008/2009
Figure 7. Total number of build-on notes created Group 1, Mr. Singh’s Pain, Part 2, 2008/2009
108
Figure 8. Social network analysis of who built-on whose notes, Group 1, Mr. Singh’s Pain, Part
2, 2008/2009
Figure 9. Total number of notes read, Group 1, Mr. Singh’s Pain, Part 2, 2008/2009
109
Figure 10. Social network analysis of who read whose notes, Group 1, Mr. Singh’s Pain, Part 2,
2008/2009
6.3.1.3 Group 1, Mary’s Misery, Pain Module 3. Results of discourse space analysis of the
third pain module, Mary’s Misery, for Group 1 indicated a total of 78 notes were created, over a
one-month period, by one facilitator and nine participants; build-on notes and reading were again
very strong (Figures 11, 12, 14).
Social network analysis of participant build-on activity reflects similar distribution; network
edges measured 21 (Figure 13). Facilitator contributions decreased in comparison to previous
modules. One particular participant’s activity has increased to become similar to that of the
facilitator. This is a positive transition indicating a shift in locus of power. Social network
analysis of who read whose notes, again, was distributed with network edges measuring 91
(Figure 15).
110
Figure 11. Total note contributions, Group 1, Mary’s Misery, 2008/2009
Figure 12. Build-on note contributions, Group 1, Mary’s Misery, 2008/2009
111
Figure 13. Social network analysis of who built-on whose notes, Group 1, Mary’s Misery,
2008/2009
Figure 14. Total number of notes read, Group 1, Mary’s Misery, 2008/2009
112
Figure 15. Social network analysis of who read whose notes, Group 1, Mary’s Misery,
2008/2009
6.3.1.4 Group 1, Judy’s Last Days, Part 1. Contribution results of discourse space analysis of
the fourth online module, Judy’s Last Days, Part 1, for Group 1 indicated 65 notes were created,
over a one-month period, by one facilitator and nine participants (Figure 16). Facilitator
contributions and build-ons were twice as high in comparison to other participants (Figures 16,
17); similar to other modules, reading results were evenly distributed (Figure 19).
Social network analysis of participant build-on activity reflects similar distribution and network
edges measured 14 (Figure 18) Social network analysis of who read whose notes, again, was
distributed with network edges measuring 76 (Figure 20), a decrease in comparison to the
previous module. This is probably due to decrease in number of network participants.
113
Figure 16. Total note contributions, Group 1, Judy’s Last Days, Part 1, 2008/2009
Figure 17. Build-on note contributions, Group 1, Judy’s Last Days, Part 1, 2008/2009
114
Figure 18. Social network analysis of who built-on whose notes, Group 1, Judy’s Last Days, Part
1, 2008/2009
Figure 19. Total number of notes read, Group 1, Judy’s Last Days, Part 1, 2008/2009
115
Figure 20. Social network analysis of who read whose notes, Group 1, Judy’s Last Days, Part 1,
2008/2009
6.3.1.5 Group 1, Judy’s Last Days, Part 2. Contribution results of discourse space analysis of
the fifth online module, Judy’s Last Days, Part 2, for Group 1 indicated 65 notes were created,
over a one-month period, by one facilitator and nine participants (Figure 21). Forty-five build-on
notes were created (Figure 22). Note contributions and reading were distributed in comparison to
the previous module (Figures 21, 24); with reading results again reaching a very high level.
Social network analysis of participant build-on activity reflects similar distribution and increase
in network edges to 20 (Figure 23). Social network analysis of who read whose notes, again, was
distributed with network edges measuring 66 (Figure 25), a slight decrease in comparison to the
previous module.
116
Figure 21. Total note contributions, Group 1, Judy’s Last Days, Part 2, 2008/2009
Figure 22. Build-on note contributions, Group 1, Judy’s Last Days, Part 2, 2008/2009
117
Figure 23. Social network analysis of who built-on whose notes, Group 1, Judy’s Last Days, Part
2, 2008/2009
Figure 24. Total number of notes read, Group 1, Judy’s Last Days, Part 2, 2008/2009
118
Figure 25. Social network analysis of who read whose notes, Group 1, Judy’s Last Days, Part 2,
2008/2009
6.3.2 Group 2, 2008/2009 ATK and Social Network Assessment Measures
Group 2 discourse space contribution results and social network analysis of the same five
modules, Mr. Singh’s Pain, Parts 1 and 2, Mary’s Misery, and Judy’s Last Days, Parts 1 and 2,
were analyzed and are presented below.
6.3.2.1 Group 2, Mr. Singh’s Pain, Part 1. Contribution results of various analysis of the first
pain module, Mr. Singh’s Pain, Part 1 for Group 2 demonstrated a moderate level of activity.
Sixty-nine notes were contributed to the discourse over a one-month period, by one facilitator
and seven participants (Figure 26). Each participant created between 1 and 12 notes. The
facilitator contributed 26—more than twice as many as the most active participant. The number
of build-on notes created was 58 (Figure 27).
Social network analysis of participant build-on activity is represented below (Figure 28). The
dominance of the number of facilitator discourse notes can be readily seen in the pattern of
distribution; network edges measured 18. Read activity by the group was extremely high (Figure
119
29). Similar to Group 1, the facilitator contributed approximately twice as many notes as other
participants. The number of notes read was very high and evenly distributed across all
participants. Social network analysis of who read whose notes is represented through the
distribution of activity; network edges measure 67 (Figure 30).
Figure 26. Total note contributions, Group 2, Mr. Singh’s Pain, Part 1, 2008/2009
Figure 27. Total number of build-on notes created Group 2, Mr. Singh’s Pain, Part 1, 2008/2009
120
Figure 28. Social network analysis of who built-on whose notes, Group 2, Mr. Singh’s Pain, Part
1, 2008/2009
Figure 29. Total number of notes read, Group 2, Mr. Singh’s Pain, Part 1, 2008/2009
121
Figure 30. Social network analysis of who read whose notes, Group 2, Mr. Singh’s Pain, Part 1,
2008/2009
6.3.2.2 Group 2, Mr. Singh’s Pain, Part 2. Results of discourse space analysis of the second
pain module for Group 2 contributions indicated 60 notes were created, over a one-month period,
by one facilitator and eight participants (Figure 31). Facilitator activity was extremely high and
participant activity was relatively low. Number of build-on notes created was 48 (Figure 32).
Social network analysis of participant build-on activity reflects similar distribution; network
edges increased to 21 (Figure 33). Group read activity was extremely high (Figure 34); most
participants read most notes. Facilitator contributions remained high in comparison to the
previous module. Similarly, social network analysis of who read whose notes represents
distributed activity. Network edges measured 55, similar to the previous module (Figure 35).
122
Figure 31. Total note contributions, Group 2, Mr. Singh’s Pain, Part 2, 2008/2009
Figure 32. Total number of build-on notes created Group 2, Mr. Singh’s Pain, Part 2, 2008/2009
123
Figure 33. Social network analysis of who built-on whose notes, Group 2, Mr. Singh’s Pain, Part
2, 2008/2009
Figure 34. Total number of notes read, Group 2, Mr. Singh’s Pain, Part 2, 2008/2009
124
Figure 35. Social network analysis of who read whose notes, Group 2, Mr. Singh’s Pain, Part 2,
2008/2009
6.3.2.3 Mary’s Misery, Pain Module 3, Group 2. Contribution results of discourse space
analysis of the third pain module, Mary’s Misery, for Group 2 indicated 55 notes were created,
over a one-month period, by one facilitator and seven participants (Figure 36). Facilitator activity
decreased in relationship to participant activity. Forty-three build-on notes were created (Figure
37). Social network analysis of participant build-on activity reflects similar distribution; network
edges decreased to 12 (Figure 38). Total read activity was strong (Figure 39). Social network
analysis of who read whose notes remained almost the same as the previous module and
demonstrated strong distribution; network edges measured 54 (Figure 40).
125
Figure 36. Total note contributions, Group 2, Mary’s Misery, 2008/2009
Figure 37. Build-on note contributions, Group 2, Mary’s Misery, 2008/2009
126
Figure 38. Social network analysis of who built-on whose notes, Group 2, Mary’s Misery,
2008/2009
Figure 39. Total number of notes read, Group 2, Mary’s Misery, 2008/2009
127
Figure 40. Social network analysis of who read whose notes, Group 2, Mary’s Misery,
2008/2009
6.3.2.4 Judy’s Last Days, Part 1, Group 2. Analysis of the fourth Group 2, online module,
Judy’s Last Days, Part 1, indicated that 36 notes were created, over a one-month period, by one
facilitator and four participants (Figure 41); 28 build-on notes were created (Figure 42).
Facilitator contributions were high in comparison to other participants. Social network analysis
of participant build-on activity reflects similar distribution; network edges measured 6 (Figure
43). Total read activity was satisfactory (Figure 44). Social network analysis of who read whose
notes, again, was distributed with network edges measuring 17 (Figure 45). More reading than
writing was event in this module. Twelve participants read the notes of others while only four
contributed.
128
Figure 41. Total note contributions, Group 2, Judy’s Last Days, Part 1, 2008/2009
Figure 42. Build-on note contributions, Group 2, Judy’s Last Days, Part 1, 2008/2009
129
Figure 43. Social network analysis of who built-on whose notes, Group 2, Judy’s Last Days, Part
1, 2008/2009
Figure 44. Total number of notes read, Group 2, Judy’s Last Days, Part 1, 2008/2009
130
Figure 45. Social network analysis of who read whose notes, Group 2, Judy’s Last Days, Part 1,
2008/2009
6.3.2.5 Judy’s Last Days, Part 2, Group 2. Contribution results of discourse space analysis of
the fifth Group 2, online module, Judy’s Last Days, Part 2, indicated 31 notes were created, over
a one-month period, by one facilitator and three participants (Figure 46). Twenty build-on notes
were created (Figure 47). Contributions were distributed in comparison to the previous module.
Social network analysis of participant build-on activity reflects similar distribution; network
edges measured 7 (Figure 48). Total read activity was extremely high (Figure 49). Seven
participants, in addition to the facilitator read almost all notes. Social network analysis of who
read whose notes, again, was distributed with network edges measuring 31 (Figure 50), a
decrease in comparison to the previous module.
131
Figure 46. Total note contributions, Group 2, Judy’s Last Days, Part 2, 2008/2009
Figure 47. Build-on note contributions, Group 2, Judy’s Last Days, Part 2, 2008/2009
132
Figure 48. Social network analysis of who built-on whose notes, Group 2, Judy’s Last Days, Part
2, 2008/2009
Figure 49. Total number of notes read, Group 2, Judy’s Last Days, Part 2, 2008/2009
133
Figure 50. Social network analysis of who read whose notes, Group 2, Judy’s Last Days, Part 2,
2008/2009
6.3.3 Social Network Pattern Analysis Across Modules and Comparatively Between
Groups, 2008/2009
New KF graphical tools enable new types of data representation and demonstration of
relationships. Visually demonstrated below are balances and shifts in traditional student/teacher
power relationships, allowing us to see the democratization of a knowledge building
environment as participants assume greater responsibility in the discourse design space. The first
analysis below (Figures 51, 52, 53, 54, 55) is composed of the 2008/2009 Group 1 social
network analysis of who built on (blue visualizations) and who read whose notes (green
visualizations).
Significant shifts are seen in who built on whose notes between the first two modules. In Mr.
Singh’s Pain, Part 1 the dominant patterns are fan-shaped and arch-shaped. The fan-shape is
indicative of the strong converging focus on facilitator discourse and conversely facilitator
finger-like extensions to participants. The arch-shape are indicative of equally strong
interparticipant discourse. These strong, definitive fan and arch patterns change and become
134
more diffuse over each successive module, until we find strong patterns, voices of many
participants equal to or stronger than the facilitators’. By Judy’s Last Days, Part 2 the patterns of
interactivity are well distributed, indicative of a shift in power structure towards one that is
maintained by all. Review of database note contributions will verify expected balance between
belief- and design-mode work and emergent ideas based on ideas brought into the discourse by
participants. Patterns of distribution of who read whose notes (green visualization) remains
highly active and stable across the first three modules and weakens in the last two modules, due
to the reduced number of participants. Summative survey comments indicates drop-out rates in
the last two modules may be due to the length of the course, from October to April.
Figure 51. Group 1, Mr. Singh’s Pain, Part 1
Figure 52. Group 1, Mr. Singh’s Pain, Part 2
135
Figure 53. Group 1, Mary’s Misery
Figure 54. Group 1, Judy’s Last Days, Part 1
Figure 55. Group 1, Judy’s Last Days, Part 2
Note. Figures 51–55: 2008/2009, Group 1, social network activity visualizations across all modules of who built on
whose notes (blue visualizations) and who read whose notes (green visualizations).
136
Group 2 social network visualization patterns were also analyzed to determine balance of power
and shifts across modules (Figures 56, 57, 58, 59, 60). The visual patterns for this group are
different from the first group. Discourse build-on notes in the first module are reasonably
distributed across facilitator and a few strong participants. Social network visualization of who
read whose notes is more active than build-on contributions. Therefore more participants are
reading than building-on. Although actual numbers of participants in Group 2 diminish in each
module as the program continues on, distribution of build-on notes remain quite evenly spread
across participants and the facilitator. In the final two modules one particular participant appears
to dominate, surpassing the facilitator, appearing to have taken on the strongest position in the
discourse, and potentially the role of the facilitator. Triangular patterns characters this visual
analysis, which is indicative of a non-centralized, distributed, and democratic environment,
primarily shared between two participants and the facilitator. Although both build-on and read
activity and interactivity appears weak, particularly in the final two modules, this may only be
due to the decrease in number of participants, not in the interest in ideas in the database. The
level of reading relative to the number of participants is high throughout.
Figure 56. Group 2, Mr. Singh’s Pain, Part 1
137
Figure 57. Group 2, Mr. Singh’s Pain, Part 2
Figure 58. Group 2, Mary’s Misery
Figure 59. Group 2, Judy’s Last Days, Part 1
138
Figure 60. Group 2, Judy’s Last Days, Part 2
Note. Figures 56–60: 2008/2009, Group 2, social network activity visualizations across all modules of who built on
whose notes (blue visualizations) and who read whose notes (green visualizations).
6.3.4 Summary of Social Network Analysis and Knowledge Forum Tools
The high level of sustained activity, by Group 1, throughout the five modules and over a five-
month period of online engagement is notable. Deep engagement, commitment to intentional
learning, and the development of a knowledge building community becomes apparent through
these visualizations of notes contributed, notes read, and social network relationships.
Social network patterns were different in Groups 1 and 2. The visual display of these results
made it easy to compare different relationships between participants and facilitator, changes
across time, and demonstrate comparative aspects between groups. It can be quickly understood
through comparative visual representation of data results that Group 2 with fewer participants
than Group 1 were not as active, but yet worked more democratically from the initial module,
which may reflect on both the facilitator and group participants. It would be reasonable to expect
that all groups will have their own characteristics and distribution patterns.
The Knowledge Building assessment visualization tools represent activity and interactivity in
new ways, beyond numbers, and thereby enable new types of analyses of patterns and
relationships across time, shifts in dynamics, and comparisons to be understood. These
visualizations provide compelling evidence in a new-found ways, important to research, the
advancement of ideas, and perhaps most profoundly for participant use for collective and self-
assessment, as embedded concurrent feedback.
139
However, the Knowledge Forum visualization tools and visual analyses, like statistical tools and
numerical analyses, have their limitations. They do not provide us with an understanding of the
discourse itself. For example, we do not know how participants worked with knowledge between
themselves or what role the facilitator played. Did the facilitator use a didactic or Socratic
approach? Did participants question each other? Did they cite readings from the digital library
and/or refer to their own experiences in practice? If misconceptions emerge in the discourse were
they dealt with by the collective, or by the facilitator, or both? Did participants contribute their
own emergent ideas and work in design-mode, as well as belief-mode? Answers to these
questions can be revealed through analysis of on the collective online discourse itself.
Participant ideas and understandings are made visible through their online note contributions.
Qualitative analysis of discourse within notes, between notes, and across notes, would enables us
to further understand important dimensions of online cognitive collective activity and
interactivity, how ideas and misconceptions are worked with, and other aspects of knowledge
building. Qualitative analysis of note content and latent semantic analyses are beyond the scope
of this thesis.
However, changes seen in network distribution patterns between facilitator and participants will
be verified through a review of the 2008/2009 Group 1 collective discourse notes in each
module. Social network distribution patterns across modules over time have enabled us to
determine how student/facilitator and student/student interactions changed across all five
modules. Results of more detailed social network analysis and a review of discourse notes will
help explain changes in patterns and verify interpretations of the current analysis to help us
unravel representations of design mode and belief mode co-creation of knowledge.
6.4 Results for Groups 1 and 2 of 2-Way ANOVA of 2008/2009
Pain Knowledge Pre-/Posttests
Consistent strong outcomes in knowledge improvement, effect, and online collective
activity/interactivity, as well as extremely positive attitudes of communal value of collective
engagement for online knowledge building in this CME course were demonstrated across all
years of this study. A closer examination of the social networks of different groups using the
same cases and comparison of physician/students and facilitator/palliative care experts
interactivity patterns is required to better understand the structural relationships that support
140
knowledge improvement. The 2008/2009 cohort year was chosen for detailed structural social
network analyses.
The 2008/2009 cohort was typical in that their mean pretest score was 67% (slightly lower but
similar to the previous 2-years). However, this two-group cohort demonstrated the largest
increase in pre-/posttest scores and effect size during the five years of this study and was
therefore, selected for further detailed social network structural and semantic analyses of
Knowledge Building relationships. Although clinical cases were the same and participant
demographics and facilitator demographics were very similar across years, online note creation
activity was substantially higher in Group 1 than in Group 2.
Therefore, in the next research subquestion, I asked: Is there a significant difference in
2008/2009 Group 1 and Group 2 pain knowledge scores from pre- to posttest and if so, is there a
significant difference between groups and effect size? A 2-way ANOVA was performed in SPSS
to determine if there is a significant difference from pre- to posttest by group and between
groups. Results are indicated in Table 19.
Overall, the difference between the 2008/2009 matched pre- and posttest means combined over
group was significant, F (1,16) = 17.97, p < .001. Overall difference between groups was not
significant. However, both groups demonstrated a similar strong effect size. Group 1
demonstrated a 16% improvement in total pain knowledge, from 68% on pretest to 84% on
posttest. In comparison Group 2 scores increased 12% from 66% on pretest to 78% on posttest.
These noteworthy differences in knowledge improvement between 2008/2009 Groups 1 and 2
demand further examination.
Table 19
Results of 2-way ANOVA Groups 1 and 2, 2008/2009
Pre Post
Groupa M (SD) 95% CI
M (SD) 95% CI Difference d
b
1 0.68 (0.14) [0.59, 0.77] 0.84 (0.12) [0.59, 0.76] 0.16 1.11
2 0.66 (0.11) [0.76, 0.92] 0.78 (0.09) [0.71, 0.86] 0.12 1.12
Note. The difference between the pre- and posttest means combined over group was significant, F (1,15) = 17.94,
p < .001. a Group 1, n = 8; Group 2, n = 9.
b Cohen’s measure of effect size.
141
Although both groups demonstrated statistically significant knowledge gains, Group 1 gains
were somewhat larger than those in Group 2. Cases and learning objectives were the same,
student incoming knowledge and facilitator content expertise were similar. Differences in
knowledge improvement between groups may be explained by further examination of collective
social network interactions, by distinguishing between facilitator and student patterns of
participation and work with ideas, as well as by detailed exploration of individual relationships
within the social network.
6.5 Social Network Structural Analysis Results 2008/2009
Pain knowledge improved significantly in both groups; however, knowledge increase was greater
in Group 1 than in Group 2, which lead us to the next and fifth research subquestion: What are
the social network structural differences between groups that supported increased knowledge
improvement and how are these differences related to centralization and/or democratization of
participation and ideas?
Investigation of this subquestion entailed comparative analyses of facilitator and student
interactivity across modules and relation to various knowledge building and social network
measures. Analyses were performed on 2008/2009 Group 1 and Group 2, with facilitator and
without facilitator (students only), and in relation to who built-on whose notes and who read
whose notes using Netminer 3 software.
The following structural social network analyses were conducted:
· Number of network links and network density;
· Social network centrality measures, specifically Eigenvector centrality, in-degree
centrality, and out-degree centrality;
· Clique members and clique cohesion index; and
· Clique characteristic, specifically total number of cliques, average size of cliques,
mean clique cohesion index, and number of cliques the facilitator belongs to.
Network position (centrality of facilitator/students and distribution of students) and relationship
to network power (prestige and dominance/democracy) although related to social network
structural analysis will be considered in a separate section and linked to sociocognitive dynamics
of the network.
142
6.5.1 Structural SNA: Network Links and Network Density Results
All social network analyses were conducted on both groups, once including the Facilitator (+F)
and once without the Facilitator (–F), students only. This analysis enabled us to distinguish
power relationships, such as facilitator dominance and student network strength (Tables 20, 21,
22).
Table 20
Number of Edges and Network Density of Build-on Notes With and Without Facilitator,
Group 1
Group 1 Who built-on whose notes Who read whose notes
Facilitator
inclusion
Number
of nodes
Number of
links/edges
Network
density
Number of
links/edges
Network
density
Mr. Singh’s Pain, Part 1
+F 11 38 0.345 93 0.845
–F 10 21 0.233 74 0.822
Mr. Singh’s Pain, Part 2
+F 11 38 0.336 110 1.000
–F 10 22 0.244 90 1.000
Mary’s Misery
+F 11 30 0.264 108 0.982
–F 10 17 0.189 88 0.978
Judy’s Last Days, Part 1
+F 11 24 0.191 65 0.591
–F 10 11 0.100 49 0.544
Judy’s Last Days, Part 2
+F 11 28 0.255 63 0.573
–F 10 14 0.156 48 0.533
Total number of edges and mean network density (SD)
+F 158 0.278 (0.06) 439 0.798
(0.21)
–F 75 0.184 (0.59) 349 0.783
(0.22)
Note. Calculations including facilitator are marked +F; without facilitator are marked –F.
143
Table 21
Number of Edges (Links) and Network Density of Build-ons With and Without Facilitator, Group
2
Group 2 Who built-on whose notes Who read whose notes
Facilitator
inclusion
Number of
nodes
Number of
links/edges
Network
density
Number of
links/edges
Network
density
Mr. Singh’s Pain, Part 1
+F 10 23 0.256 80 0.875
–F 9 11 0.153 63 0.700
Mr. Singh’s Pain, Part 2
+F 10 28 0.300 81 0.900
–F 9 15 0.194 64 0.889
Mary’s Misery
+F 10 15 0.156 71 0.789
–F 9 5 0.069 55 0.764
Judy’s Last Days, Part 1
+F 10 10 0.100 37 0.411
–F 9 3 0.280 15 0.194
Judy’s Last Days, Part 2
+F 10 11 0.100 28 0.311
–F 9 5 0.056 18 0.250
Total # of Edges
+F 87 297
–F 39 215
Mean Network Density (SD)
+F 0.182 (0.09) 0.657 (0.28)
Note. Calculations including facilitator are marked +F; those without facilitator are marked –F.
144
Table 22
Summary of Edges/Links and Network Density: Groups 1 and 2
Who Built-On Whose Notes Who Read Whose Notes
Group Facilitator Number
of Nodes
Total Number
of Links/Edges
Mean (SD)
Network Density
Percentage
Total Number
of Links/Edges
Mean (SD)
Network Density
Percentage
1 +F 11 158 0.278 (0.06) 439 0.798 (0.21)
2 +F 10 87 0.182 (0.09) 297 0.657 (0.28)
1 –F 10 75 0.184 (0.59) 349 0.783 (0.22)
2 –F 9 39 0.150 (0.09) 215 0.559 (0.32)
Note. Calculations including facilitator are marked +F, without facilitator are marked –F. Group 1 (n = 11); Group
2 (n = 10)
Comparison between groups (with facilitator) was conducted. Groups 1 mean network density of
build-ons (0.278) was higher than Group 2 (0.182). This is related to the high number of links or
edges created by the Group 1 participants with facilitator. High density measures of notes read
are demonstrated by both groups; however again Group 1 measures (0.798) are higher than
Group 2 (0.657).
Comparison between student groups (without facilitator) was also conducted. Both student-only
groups demonstrated network strength in terms of mean density when the facilitator was
removed. Group 1 build-on network density with facilitator measured 0.278 as opposed to
student only 0.184. Similarly Group 2 build-on network density with facilitator measured 0.182
as opposed to student only, which measured 0.150. Network density of who read whose notes
remained strong.
6.5.1.1 Network density results of t-tests and effect. Results of t-tests on density and effect
size across all five modules with and without Facilitator demonstrate no significant difference
and no strong effects (Table 23). Therefore, it can be concluded that density of student notes
built-ons and notes read were relatively strong in both Groups 1 and 2.
145
Table 23
Comparison of Density with and Without Facilitator of Notes Built-On and Read
Group t df p Cohen’s d
Density (built-on)† 1 .501 9 .628 0.29
Density (read) 1 .160 19 .875 0.07
Density (built-on) 2 .774 17 .450 0.11
Density (read) 2 .712 17 .486 0.33
Note. Group 1 (n = 11); Group 2 (n = 10).
†Probability is corrected for unequal group variances.
In summary, social network densities in both groups were maintained with and without the
facilitator demonstrating power of the student network and active engagement in both building-
on each other’s ideas and/or reading each other’s notes. Social network change across modules
within groups, exemplifies the dynamic flow of sustained work within networks across time.
6.5.2 Social Network: Eigenvector, In-degree, and Out-degree Centrality Results
Results of Eigenvector centrality, in-degree, and out-degree centrality, with and without
facilitator, across all modules, for Group 1 are presented in Table 24 and for Group 2 are
presented in Table 25.
Mean Eigenvector centrality measures of build-ons and notes read are low in both Groups 1 and
2. These low measures are indicative of well-distributed groups. Eigenvector analysis of groups
with and without facilitators demonstrates little change and is therefore indicative of strong
student networks. In-degree/out-degree measures of Group 1 and Group 2 build-ons dropped
markedly in some modules when the facilitator was removed, indicative of a high degree of
facilitator participation.
146
Table 24
Social Network Centrality Measures (in Percentages) With and Without Facilitator, Group 1
Group 1 Who built-on whose notes Who read whose notes
Facilitator
inclusion
Eigenvector
centrality (SD)
In-degree
centrality
Out-degree
centrality
Eigenvector
centrality (SD)
In-degree
centrality
Out-degree
centrality
Mr. Singh’s
Pain, Part 1
+F 23.0 (19.5) 61.0 50.0 27.5 (12.3) 17.0 6.0
–F 23.8 (20.8) 35.8 23.5 29.2 (12.0) 19.8 7.4
Mr. Singh’s
Pain, Part 2
+F 24.6 (17.5) 29.0 62.0 28.7 (9.4) 0.0 0.0
–F 27.7 (15.3) 34.6 59.3 30.5 (8.2) 0.0 0.0
Mary’s
Misery
+F 25.3 (16.4) 37.0 37.0 28.9 (8.7) 2.0 2.0
–F 26.9 (16.7) 40.7 40.7 30.5 (8.5) 2.5 2.5
Judy’s Last
Days, part 1
+F 21.9 (20.8) 34.0 56.0 26.2 (14.8) 34.0 23.0
–F 21.2 (23.4) 25.9 13.6 27.9 (15) 25.9 25.9
Judy’s Last
Days, Part 2
+F 23.6 (18.7) 49.0 49.0 26.6 (14.1) 25.0 25.0
–F 24.1 (20.5) 19.8 44.4 28.1 (14.5) 27.2 27.2
Mean (SD)
+F
23.7
(1.3)
42.0
(12.9)
50.8
(9.3)
27.6
(1.2)
15.6
(14.6)
11.2
(11.9)
–F
24.7
(2.61)
31.4
(8.4)
36.3
(18.0)
29.2
(1.3)
16.2
(14.8)
12.6
(13.0)
Note. In-degree centrality (receiving links) and out-degree centrality (linking to others); all centrality measures are
interpreted as the higher the measure the more centralized the network, the less distributed. Calculations including
facilitator are marked +F, without facilitator –F. Group 1 (n = 11); Group 2 (n = 10).
147
Table 25
Social Network Centrality Measures (in Percentages) With and Without Facilitator, Group 2
Group 2 Who built-on whose notes Who read whose notes
Facilitator
inclusion
Eigenvector
centrality (SD)
In-degree
centrality
Out-degree
centrality
Eigenvector
centrality (SD)
In-degree
centrality
Out-degree
centrality
Mr. Singh’s Pain, Part 1
+F 23.1 (21.6) 33.3 58.0 29.6 (11.2) 12.4 12.4
–F 23.8 (23.3) 25 25 31.8 (10.1) 14.1 14.1
Mr. Singh’s Pain, Part 2
+F 24.6 (17.5) 29.0 62.0 29.6 (11.3) 11.1 11.1
–F 27.7 (15.3) 34.6 59.3 32.7 (6.5) 12.5 12.5
Mary’s Misery
+F 23.1 (16.4) 44.4 32.2 29.9 (10.4) 23.5 11.1
–F 24.8 (22.2) 20.3 20.3 32.4 (7.8) 26.6 12.5
Judy’s Last Days, part 1
+F 21.9 (20.8) 34.0 56.0 26.7 (17.0) 65.4 16.1
–F 18.4 (27.8) 25.0 10.9 26.9 (19.7) 34.4 20.3
Judy’s Last Days, Part 2
+F 19.4 (25.0) 25.9 25.9 27.2 (16.2) 51.9 14.8
–F 19.0 (27.4) 21.9 21.9 28.6 (17.2) 56.3 14.1
Mean (SD)
+F
22.4
(1.9)
33.3
(7.0)
46.8
(16.5)
28.6
(1.5)
32.8
(24.5)
13.1
(2.2)
–F
22.7
(4.0)
25.4
(5.5) 27.5
(18.5)
30.5
(2.6)
28.8
(17.8)
14.7
(3.2)
Note. In-degree centrality (receiving links) and Out-degree centrality (linking to others); all centrality measures are
interpreted as the higher the measure the more centralized the network, the less distributed. Calculations including
facilitator are marked +F, without facilitator –F. Group 1 (n = 11); Group 2 (n = 10).
6.5.2.1 t-test and effect size results of social network centrality. Significant difference and
large effect size were found for Group 1 when tested with and without Facilitator, on social
network centrality dimensions of in-degree and out-degree of notes built-on (Table 26).
Therefore we can conclude the Facilitator in Group 1 was highly participative in terms of
receiving and linking or building-on others notes and had a large effect.
148
Table 26
Comparisons of SN Centrality Measures With and Without Facilitator, by Group, Across All
Modules
Group t df
a p Cohen’s d
Eigenvector (B) † 1 1.16 13 .269 0.51
In-degree (B)* 1 2.21 19 .039 0.98
Out-degree (B) †* 1 2.29 13 .039 1.01
Eigenvector (R)** 1 3.09 19 .006 1.35
In-degree (R) 1 .09 19 .933 0.04
Out-degree (R) 1 .26 19 .801 0.11
Eigenvector (B) † 2 .22 11 .830 0.10
In-degree (B)* 2 2.73 17 .014 1.26
Out-degree (B)* 2 2.41 17 .028 1.10
Eigenvector (R) 2 1.96 17 .067 0.89
In-degree (R) 2 .41 17 .686 0.19
Out-degree (R) 2 -1.26 17 .223 0.57
Note. B = Built-on notes; R = notes Read. Group 1 (n = 11); Group 2 (n = 10). Results preceded by a negative signs
indicate that the Group 2 mean exceeded the Group 1 mean. a Because data were available only in aggregated form, grouped t-tests were used.
†Probability is corrected for unequal group variances. * p < .05, ** p < .01.
Significant difference and a very large effect size (1.35) was also found for Eigenvector notes
read, with and without Facilitator for Group 1. Therefore the Facilitator was also a strong
participant in reading group notes. Although there was not a significant difference in Eigenvector
notes read, with and without Facilitator for Group 2, the effect size was large (0.89). Therefore it
can be said that the Facilitator of Group 2 was a very active reader.
Like Group 1, Group 2 demonstrated significant difference and large effect sizes on differences
with and without Facilitator for dimensions of in-degree and out-degree, of notes built-on, across
all modules. Therefore the Facilitator in this group could also be considered a very active
member.
149
6.5.2.2 t-test and effect size results of social network density and centrality measures
across groups. Comparison results between Groups 1 and 2, with a facilitator, across all five
modules indicates that density of notes built-on is significantly different (Table 27). Difference
in density for notes built-on was significantly stronger in Group 1 (p < .01) and effect size (1.31).
Therefore we can conclude that substantially more notes were built-on in Group 1 and student
and facilitator participation was stronger and significantly different from Group 2. Density of
build-on notes appears to be the major factor emerging from social network analyses in this case
study that is related to higher levels of knowledge improvement in Group 1, across all modules.
Table 27
Comparisons of SN Measures, between Groups 1 and 2, With Facilitator, Across All Modules
t df p d
Density (Build-ons)** 2.90 19 .009 1.31
Density (Read) 1.31 19 .205 0.57
Eigenvector (B) 1.75 19 .096 0.76
In-degree (B) † 1.93 16 .072 0.83
Out-degree (B) † 0.67 14 .514 0.30
Eigenvector (R) -1.71 19 .104 -0.74
In-degree (R) † -1.93 14 .073 -0.85
Out-degree (R) † -0.52 11 .616 -0.22
Note. B = Built-on, R = Read. Group 1 (n = 11); Group 2 (n = 10). Results preceded by a negative signs indicate that
the Group 2 mean exceeded the Group 1 mean.
†Probability is corrected for unequal group variances. * p < .05, ** p < .01.
All other social network dimensions, of notes build-on and notes read, do not demonstrate
significant difference or effect size between groups. In other words, Groups 1 and 2 are not
significantly different in terms of various centrality measures, i.e. Eigenvector centrality,
receiving notes and linking of notes, both in terms of notes built-on and notes read. Therefore we
can conclude both Groups 1 and 2 were well-distributed, non-centralized networks.
150
6.6 Social Network Analyses Results of Cliques (K)
and Cohesion Index of Build-On Ideas Only
Since social network results of who read whose notes was equally strong across both groups,
further structural analyses of cliques and cohesion measures will focus on detailed examination
of building-on ideas only. In addition, analyses will now be performed only with inclusion of the
facilitator to determine the relationship between facilitator and student interactions.
Results of clique members and cohesion index by module and across groups can be found in
Tables 28 and 29. Although Group 1 and Group 2 clique sizes were similarly composed of three
to four participants, there are marked distinctions between the structures of cliques across
groups. Group 1 had almost twice as many total cliques (27) as Group 2 (14).
Most striking is the fact that the Group 1 facilitator participated in all 27 cliques, which is more
than three times as many as the Group 2 facilitator, who participated in eight of 14 cliques.
Despite the very high level of Group 1 facilitator engagement the Cohesion Index across
modules was lower than Group 2, indicative of a more highly distributed and inclusive social
networks. Interclique connectivity was maintained in both groups by the numerous participants
that were active across many cliques. “This ‘bridging’ phenomenon provides for a wealth of
information flowing to all members” (Aviv et al., 2003, p. 13).
Important findings emerged from this analysis. Findings of high-level facilitator participation
within well-distributed network provokes new questions, particularly in regards to scaffolding
and supporting enhanced knowledge improvement, as was found in Group 1.
151
Table 28
Clique Members and Cohesion Index Results of Build-Ons by Group With Facilitator
Group 1 Group 2
Number of
Cliques (K) in
each Module
Members Clique
Cohesion Index
Members Clique
Cohesion Index
Mr. Singh’s
Pain, Part 1
K1 11,6,7,9,10,5 3 2,6,9,1 3
K2 11,6,7,9,10,1 3.333 2,6,9,4 3
K3 11,6,3 2 2,6,10 2.625
K4 11,2,5 2.182 2,5,3 3
Mr. Singh’s
Pain, Part 2
K1 9,11,7,5,10 2.143 10,3,5,7,1 2.778
K2 9,11,7,6 1.647 10,3,5,2 2
K3 9,11,7,8 1.867 10,3,5,8 1.846
K4 9,11,3,6 2 10,3,9,8 2.4
K5 9,11,1,5 2
K6 9,11,2,6 1.867
K7 9,11,2,10 1.867
Mary’s Misery
K1 9,11,10,6 2.333 3,10,8,1 4.8
K2 9,11,10,7 2.154 3,10,5 3.5
K3 9,11,10,5 2 9,1,10 3.5
K4 9,11,1 2
K5 9,11,3,5 2.333
K6 8,11,7 2.667
K7 8,11,5 2.4
Judy’s Last
Days, Part 1
K1 11,7,8,9 4.667 10,3,2 7
K2 11,7,1 3 10,3,5 7
K3 11,3,2 4.8
K4 11,6,9 3
K5 11,6,1 3.429
Judy’s Last
Days, Part 2
K1 6,11,7,8,9 3.333 1,3,5,10 n/a
K2 6,11,7,1 2.8
K3 6,11,3,8 2.8
K4 6,11,2,9 2.8
M (SD) 2.70 (0.71) 4.26 (1.93)
Note. Group 1 n = 11; Group 2 n = 10. K = Clique. Facilitators = #11 (Gr. 1) and #10 (Gr. 2).
152
Table 29
Clique and Cohesion Index Results of Build-On Notes in Groups1 and 2, With Facilitators
Who built-on whose notes
2008/2009
Group
Total number
of cliques
Average size
of cliques %
(SD)
Mean clique
cohesion index
% (SD)
No. cliques
facilitator
belongs to
Mr. Singh’s Pain, Part 1
1 4 4.50 (1.73) 2.63 (0.64) 4
2 4 3.50 (0.58) 2.91 (0.19) 1
Mr. Singh’s Pain, Part 2
1 7 4.14 (0.38) 1.91 (0.16) 7
2 4 3.50 (0.58) 2.91 (0.19) 1
Mary’s Misery
1 7 3.57 (0.53) 2.27 (0.24) 7
2 3 3 4.2 3
Subtotal # and Mean (SD)
1 18 4.07 (0.47) 2.27 (0.36) 18
2 11 3.33 (0.29) 3.34 (0.74) 5
Judy’s Last Days, Part 1
1 5 3.2 (0.45) 3.78 (0.89) 5
2 2 3 7 2
Judy’s Last Days, Part 2
1 4 4.25 (0.5) 2.93 (0.27) 4
2 1 4 n/a 1
Total # and total mean (SD)
1 27 3.932 (0.53) 2.70 (0.71) 27
2 14 3.4 (0.42) 4.26 (1.93) 8
Note. Higher Cohesion Index = more isolated cliques. Group 1 n = 10; Group 2 n = 11.
153
6.7 Relationship of Social Networks to Knowledge Improvement Scores
6.7.1 Relationship of Social Network Structural Analyses of the Three Pain Modules to
Pain Knowledge Improvement Pre-/Posttest Scores
The relationship of various dimensions of social network structural analyses were examined
alongside the results of notes built-on in three pain modules, for comparison between groups, and
with pain knowledge pre-/posttest results. Tests for significant difference and effect size were
also conducted to determine if there is a relationship between social network participation
patterns and knowledge improvement.
Results of clique and Cohesion Index (Table 30) for the three pain modules indicate that the
Group 1 facilitator participated in all 18 cliques for the three pain modules. While the Group 2
facilitator only participated in less than 50% (5/11) of the cliques. Mean clique Cohesion Index
was lower in Group 1 (2.27) than Group 2 (3.33), indicative of the fact that the group on the
whole was more distributed or democratized. Group 1 participants created approximately 40%
more links than Group 2 and correspondingly network density measures were higher in Group 1
(0.315) than Group 2 (0.237) (Table 31). Eigenvector and out-degree centrality measure are
similar between groups. In-degree centrality for Group 1 is 42.33, as compared to 27.35 for
Group 2.
Relationship patterns indicate that higher performance on pre-/post knowledge tests in Group 1
were related to higher levels of build-on notes in KF, higher number of network edges/density,
lower coherence index and centrality measures, as well as a greater number of cliques, and more
facilitator engagement in an evenly distributed network.
Group 1 network density of build-on notes demonstrated significant difference compared to
Group 2 (p <.01), as well as an extremely strong effect size of 1.40 (Table 32). Lower cohesion
of Group 1 build-ons was also statistically significant (p < .001) when compared to Group 2,
with an effect size of 1.84. In-degree centrality, receiving of links, was also significantly stronger
in Group 1 than Group 2 (p<.05), with an effect size of 1.18. Eigenvector and out-degree
centrality measures were not significantly different indicating that both groups were well-
distributed and many links between participant notes and ideas were made.
154
Table 30
Comparative Summary of SN Results of Groups 1 and 2 in Three Pain Modules
Cohesion
index % (SD)
No. of
edges/
links
Density Eigenvector centrality
%
In-degree centrality
%
Out-degree
centrality %
Mr. Singh’s Pain, Part 1
G1 2.63 (0.64) 38 0.345 23.0 61.0 50.0
G2 2.91 (0.19) 23 0.256 23.1 33.3 58.0
Mr. Singh’s Pain, Part 2
G1 1.91 (0.16) 38 0.336 24.6 29.0 62.0
G2 2.91 (0.19) 28 0.300 27.0 28.4 65.4
Mary’s Misery
G1 2.27 (0.24) 30 0.264 25.3 37.0 37.0
G2 4.2 15 0.156 24.8 20.3 20.3
Total
G1 106
G2 66
Mean (SD)
G1
2.27
(0.36)
0.315
(0.04)
24.3
(1.1)
42.3
(16.6)
49.7
(12.5)
G2 3.34
(0.74)
0.237
(0.07)
25.1
(2.0)
27.4
(6.6)
47.9
(24.2)
Note. G = Group; Group 1 n = 11, Group 2 n = 10.
Table 31
Comparisons of Social Network Measures of Build-Ons, Between Groups, Across Three Pain
Modules
t df p Cohen’s d
Cohesion Index † *** -4.15 13 .001 -1.84
Density (B) † ** 3.09 14 .008 1.40
Eigenvector (B) -1.10 19 .284 -0.48
In-degree (B) † * 2.76 13 .016 1.18
Out-degree (B)† .22 13 .841 0.09
Note. Group 1 n = 11, Group 2 n = 10. Numbers of cases differ among analyses due to uneven group participation.
Negative (-) sign before number = stronger in Group 2 than Group 1. B = Build-on.
†Probability is corrected for unequal group variances
* p < .05, ** p < .01, *** p < .001.
155
Table 32
Results of 2008/2009 Pain Pre-/Posttests
Pre Post
Groupa M (SD) 95% CI
M (SD) 95% CI Difference d
b
1 0.68 (0.14) [0.59, 0.77] 0.84 (0.12) [0.59, 0.76] 0.16 1.11
2 0.66 (0.11) [0.76, 0.92] 0.78 (0.09) [0.71, 0.86] 0.12 1.12
Note. The difference between the pre- and posttest means combined over group was significant, F (1,15) = 17.94, p
< .001. a Group 1 n = 8; Group 2 n = 9.
b Cohen’s measure of effect size.
6.7.2 Correlation Between Results of Social Network Analyses and Pain Posttest Scores
The data for Mr. Singh Parts 1 and 2 and Mary’s Misery were reformatted and read into SPSS
for Group 1. The posttest score was correlated to the Eigenvector score, the in-degree and out-
degree measures. Because of the low number of cases nonparametric correlation (Spearman’s
rho) was used. Over the three scenarios, only the in-degree measure was correlated significantly
for Mr. Singh part 1 only to posttest (r = .724, p = .042) . The remaining correlations to post (or
difference) scores were not significant at = or < .1.
For Group 2 the results with retention of zero network scores were not significant for Mr.
Singh’s Pain, Part 1. However, when student number 2 (with zero activity) was omitted, the
picture changed. Posttest was related significantly to Eigenvector (rho = .822, p = .012), to in 1
(rho = .740, p = .036), and to out-degree centrality (rho = .682, p = .063). The results for Group 2
on Mr. Singh part 2 or Mary’s Misery were not significant either with or without the cases with
zero activity.
Groups 1 and 2 were combined for further analysis. Table 33 shows nonparametric correlations
along with significance of the posttest score with the social network variables. For this analysis
the zero measures were retained. Of the nine correlations, all are positive and all are above .2, six
above .3. Two are significant at p < .05 (Eigenvector centrality in Mr. Singh’s Pain, Part 1 and
in-degree in Mary’s Misery) and 2 at p < .1 (in-degree in Mr. Singh’s Pain, Part 1 and out-degree
in Mary’s Misery).
156
Table 33
Spearman Correlations of the Post Score with Social Network Variables
Variable Eigen
Mr S 1
In
Mr S 1
Out
Mr S 1
Eigen
Mr S 2
In
Mr S 2
Out
Mr S 2
Eigen\
MM
In
MM
Out
MM
Correlation .461 .516* .262 .370 .220 .257 .399 .429 .511
*
Significance .063† .034* .309 .144 .395 .319 .112 .085† .036*
Note. * p < .05, † p <.1.
In conclusion, this analysis demonstrates important and strong relationships between Group 1
knowledge improvement pre-/posttest outcomes and social network measures of high build-on
note density, low cohesion index, low Eigenvector centrality, accompanied by significant
measures of centrality, confirming key Knowledge Building principles of the importance of
distributed networks, decentralized participation, and democratization of ideas in a communal
space.
6.8 Cluster 2 Summary of Performance Measures (Over and Above Traditional
Learning) From Social Network Structural Results
Cluster 2 examined social network structural measures, over and above and above traditional
learning measures to determine relationships between Knowledge Forum activity/interactivity,
network density, centrality, cohesion, clique measures, and their relationship to pre-/posttest
outcomes.
This cluster examined three questions:
What are the participant online activity and interactivity measures, by module, by group, and by
year, in the 2005–2009 study? The Knowledge Forum analytic toolkit was used to determine
read, write, build-on measures. Participation was high across all categories. Notes read were
particularly high. This measure may or may not be reliable. It is not known if participants read or
merely opened/closed notes without reading them. Build-ons notes are seen in this study, as the
most reliable indicator Knowledge Building.
The second question in this cluster was: Is there a significant difference in 2008/2009 Groups 1
and 2 pain knowledge scores from pre- to posttest and is there a significant difference between
157
groups? Result indicated pain knowledge improved significantly in both groups; however,
knowledge increase was greater in Group 1 than in Group 2.
Therefore, the third question asked was: What are the social network structural differences
between groups that supported increased knowledge improvement and how are these differences
related to centralization and/or democratization of participation and ideas?
Consistent with the findings of greater knowledge gains in Group 1 than in Group 2, are patterns
of higher-level build-on activity, number of edges, and network density. Social network analysis
was used to evaluate distribution of participation and ideas in each group. These analyses
demonstrated similar Eigenvector centrality of well-distributed networks in both groups, in
regards to ‘who built-on whose notes’ and ‘who read whose notes’. Mean effect of ‘statistical’
removal of facilitator in both groups was negligible, attesting to the fact that both Group 1 and
Group 2 represented structurally strong and well-distributed student networks. Cohesion Index of
differed. Mean Cohesion Index of Group 1 was substantially lower than Group 2, noting
differences between groups, across cliques, within modules.
Most remarkable is the number of cliques created by Group 1 participants, as well as the overall
number of cliques the facilitator belonged and the strong interclique participation by students.
Contrary to popular notions of democratization in education, the expert/teacher/facilitator may be
extremely active as a “partner” and co-creator of knowledge within a social network.
Atypical high levels of facilitator participation require further examination in relationship to
students within the network. This will be examined in this study using social network analysis
measures and maps for analysis of network position. Social network position structures are
related to power structures. Results of these analyses can be interpreted according to centrality of
position and power of ideas. Results of the next social network analysis will begin with
identification of who is at- the-”structural” core (facilitator and/or students), mid and periphery
and track power dynamic/flows/shifts across modules (Aviv et. al., 2003).
Results presented herein, demonstrate structural relationships are indicative of high functioning
group, where the facilitator, plays a key role in knowledge work with students. These social
network structural relationships are also indicative of Knowledge Building symmetry, collective
responsibility, and epistemic agency important to the democratization of continuing medical
education.
158
At this point, we can conclude that there is a strong relationship between increased knowledge
gains, high levels of participant online activity, and communal work that includes knowledge
experts among participants in decentralized, opportunistic environments. These relationships
may be highly integrated with notions of position and power in the social network,
democratization of ideas and Knowledge Building discourse.
As shown in this chapter, structural social network analysis helped us to explain dynamic
interactions and relationships important to the improvement of network distribution and
democratization Knowledge Building in continuing medical education. These types of social
network analyses enables us to consider how new cultural tools, such as Knowledge Forum,
enable new participant structures and interactions that have the potential to change asymmetries
and traditional pedagogic paradigms.
The current analysis also evoked further questions particularly about facilitator/student
interactions and relationships. Both facilitators were highly active in the social networks. What
we need to know is: How did they participate? Did they didactically dominate the group or
facilitate and scaffold knowledge building? In social network analysis terms, did they share the
centre space and create opportunities for democratized engagement or was the environment
autocratic, represented only by the facilitator ideas-at-the-centre? We need to know what were
the sociocognitive dynamics that enabled knowledge work over and above learning?. Results of
social network position/power analysis, as well as content analysis of the discourse will be
presented in the next chapter to help determine answers to these and other questions.
159
CHAPTER 7
RESULTS—CLUSTER 3:
SOCIAL NETWORK POSITION/POWER ANALYSIS AND SOCIOCOGNITIVE
DYNAMICS OF KNOWLEDGE BUILDING
7.1 Introduction
Chapter 7 presents results based on the third cluster of subquestions, regarding the sociocognitive
dynamics that enable work over and above traditional learning. Cluster 3 is guided by the
following research subquestion: What are the social network relationships between structural
position, power (defined as centrality of ideas), and knowledge improvement; and how are these
relationships reflected in facilitator/student sociocognitive dynamics of Knowledge Building
(beyond learning objectives), through emergent themes, in complexity of discourse, and by
indicators of Knowledge Building? Table 34 summarizes data analyses and data sources used to
determine the results herein.
Concepts of power, dually defined as structural position and centrality of ideas are examined in
the first analysis in this chapter. The centrality or power of one’s ideas are explored in relation to
knowledge improvement. Centrality maps and corresponding measures of position/power of
facilitator/students are provided across all 10 modules; results are compared across groups.
Content analysis of discourse notes was conducted on forty percent of the dataset; two modules
from each of the two groups was selected for within note analyses. Results of four analyses are
presented, including: (a) facilitator/student sociocognitive dynamics, (b) identification of
emergent themes beyond those predetermined by learning objectives, (c) complexity of clinical
discourse, and (d) Knowledge Building indicators and exemplars. Results of within note analyses
aim to further our understanding of the sociocognitive dynamics that support adductive and
abductive Knowledge Building, working with ideas more deeply and more broadly.
160
Table 34
Cluster 3: Social Network Analysis of Sociocognitive Dynamics
CLUSTER 3 SOCIOCOGNITIVE DYNAMICS THAT ENABLE WORK OVER AND ABOVE
TRADITIONAL LEARNING
Subquestion
6
What are the social network relationships between structural position, power (defined as
centrality of ideas), and knowledge improvement; and how are these relationships reflected in
facilitator/student sociocognitive dynamics of Knowledge Building (beyond learning objectives),
through emergent themes, in complexity of discourse, and by indicators of Knowledge Building?
Data Analyses Data Sources
a. NETWORK POSITION AND POWER
(measures and mapped visualizations)
Structural analysis of position/idea centrality/power
and relationship to knowledge improvement
2008/2009 Groups 1 and 2, five modules,
build-ons (with facilitator),
= 10 social network centrality maps
Identification of whose ideas are at the core, mid,
or periphery (facilitator and/or students; Aviv
et al., 2003)
Relationship of student network position to
individual pre-/posttest knowledge gain Pre-/posttest results by individual and
social network centrality maps
b. FACILITATOR/STUDENT SOCIOCOGNITIVE DYNAMICS
Facilitator stance
Facilitator as monitor, mentor, participant, or
expert (modified protocol by Tabak
&Baumgartner, 2004)
2008/2009 Groups 1 and 2, four modules
(40% of dataset) to be scored by PI (and
20% by second rater), within-note analysis
Who asks the questions that drive knowledge
work?
As above
c. ANALYSIS OF EMERGENT THEMES (OVER AND ABOVE
LEARNING OBJECTIVES)
Categorization of themes and threads based on
predefined learning objectives and emergent ideas
(Zhang et al., 2009)
2008/2009 Groups 1 and 2, four modules
(40% of dataset) to be scored by PI,
within-note analysis
Relationship of emergent themes to abductive and
adductive knowledge improvement
As above
d. ANALYSIS OF COMPLEXITY OF DISCOURSE
Use of semantic analysis of clinical discourse scale
(Bordage, 1994), 4-pt rating scale of clinical
discourse as reduced, dispersed, elaborated, or
compiled
2008/2009 Groups 1 and 2, four modules
(40% of dataset) to be scored by PI (and
30% by second rater)
e. ANALYSIS OF KNOWLEDGE BUILDING INDICATORS
Evidence of Knowledge Building principles,
particularly community, democracy, agency, and
improvable ideas related to Knowledge Building
intentionality and emergent ideas/themes
2008/2009 Groups 1 and 2, four modules
(40% of dataset) to be scored by PI (and
second rater)
Use of scoring protocol by Sibbald (2007) and
definitions by Scardamalia (2002)
Note. Extracted from Table 4, Chapter 3.
161
7.2 Results of Social Network Position/Power Analysis
Ideas of power relationships frame investigations in this next section (Foucault, 2001). The
notion of position within a social network has two sides. It may refer to the structural position an
individual may hold within the social network relative to others. It may also refer to the power or
centrality of knowledge and ideas; this interpretation can be thought of as semantic in nature and
related to content analysis, within note analysis of discourse results in the next section.
Results of social network structural position/power analyses presented in this section include:
1. Relationship of student network position to individual pre-/posttest knowledge gain;
2. Social network structural analysis results of position/power represented in centrality
maps and corresponding centrality measures.
7.2.1 Relationship of Individual Student Social Network Position Scores to Difference on
Pre-/Posttest Knowledge Scores
Position/power and knowledge improvement relationships were identified through examination
of the 2008/2009 Group 1 (Table 35) and Group 2 (Table 36) individual pre-/posttest scores and
centrality position measures.
Table 35
Group 1 Individual Differences in Student Pain Knowledge Pre-/Posttest Scores
Group 1 individual pre- and posttest scores 2008/2009
Student Pretest score Posttest score Difference
Albert 0.58 0.90 0.32
Craig 0.87 0.87 0.00
Don 0.56 0.59 0.03
Francis 0.63 n/a n/a
Gail 0.59 0.95 0.36
John 0.69 0.73 0.04
Jeff 0.83 n/a n/a
Len 0.80 0.88 0.08
Mary 0.85 0.85 0.00
Sara 0.51 0.93 0.41
162
Table 36
Group 2 Individual Differences in Student Pain Knowledge Pre-/Posttest Scores
Group 2 individual pre- and posttest scores 2008/2009
Student Pretest score Posttest score Difference
Brenda 0.57 0.85 0.28
Jack 0.66 0.83 0.16
Jenn 0.80 0.85 0.04
Jeorge 0.61 0.78 0.16
Maria 0.64 0.78 0.14
Phil 0.49 0.57 0.09
Rose 0.65 0.76 0.11
Shira 0.78 0.88 0.10
Sven 0.78 0.78 0.00
Individual pre-/posttest scores were calculated in SPSS for each student in Groups 1 and 2.
Students were categorized as greatest gainers, incoming stars, or others. All were included in the
position/power analyses. However, greatest focus was given to greatest gainers, to see what
positions they held in module and if, and how these positions changed across modules over time.
Incoming stars or those individuals that participated in the course with an incoming high level of
knowledge were particularly noteworthy; it was hypothesized that these individuals would take
on central roles, like or near the facilitator.
Three students in Group 1 and 1 student in Group 2 were identified as greatest gainers, those that
made the largest gain from pre- to posttest score. In Group 1 they were Albert (32% gain); Gail
(36% gain); and Sara (41% gain).
In Group 2 the one student was Brenda (28% gain).
Three students in Group 1 and one student in Group 2 were identified as incoming stars; their
pretests scores were 80% or higher and their posttest scores were 85% or higher. In Group 1 they
were Craig, Len, and Mary.
In Group 2 the one student was Jenn.
7.2.2 Social Network Position, and Power Maps and Measures
Social network position/power analyses were done across 2008/2009 Groups 1 and 2, on all five
online modules in each group, for a total of 10 modules. Only build-on notes were analyzed.
163
Early social network centrality analyses showed build-on notes were more variable across
groups, while notes read were very homogeneous—all at a high level. Therefore build-on notes
were chosen to examine possible differences.
The Group 1 facilitator is number 1 and the Group 2 facilitator is number 2. Facilitators are
marked by a blue square and students are marked by red dots on the centrality maps. As in other
analyses, all real names were replaced with fictitious ones. Nonparticipants in the module are
indicated in tables, but are not removed from the centrality maps.
Netminer 3 software was used to perform social network position analysis to identify who is at
the structural centre, as well as who is in midfield and who is at the periphery (Aviv et al., 2003).
“Who” is identified as “facilitator” or “student.” Five modules for each of the two groups were
mapped. Social network centrality measures were also calculated to enable interpretation of the
relationships mathematically as well as visually. Dynamic shifts in position over time, across
modules was noted. In particular, the shift of three Group 1 students to the centre and a fourth
near the threshold, dramatically displays the transition of power, epistemic agency, and the
democratization ideas in the problem space. Interesting relationships between social network
position and greatest knowledge gain were found. The results of position/power analyses, related
centrality scores, and pain pre- to posttest improvement scores are presented below for each of
the five modules in each group.
7.2.2.1 Position/power results for Group 1, Mr. Singh’s Pain, Part 1. Results indicate that
the central positions were occupied by the facilitator and 2 students, Gail (greatest gainer) and
Mary (incoming star; Figure 61). In terms of Eigenvector centrality the facilitator held the most
power. However, Eignevector, in-degree, and out-degree measures for the “greater gainer” and
“star” are very similar. This level of epistemic agency from student #5 is particularly
noteoworthy in the firt module. Ideas in the problem space were well distributed, with two
students in the centre, three midfield, and four on the periphery (see Table 37).
164
Table 37
Social Network Position/Power Map and Centrality Measures for Group 1, Mr. Singh’s Pain,
Part 1, Build-On Notes
Mr. Singh’s Pain Part 1, build-on notes
Group 1 Participant Eigenvector Centrality map position
Albert 0.107 periphery
Craig 0.06 periphery
Don 0.046 periphery
Francis 0 n/p
Gail 0.444 core
John 0.199 mid
Jeff 0.217 mid
Len 0.034 periphery
Mary 0.446 core
Sara 0.387 mid
Facilitator 1 0.591 core
Distribution 3 3 4 + 1 n/p
Note. n/p = nonparticipant.
165
Figure 61. Social network position/power map Group 1, Mr. Singh, Part 1, Build-on notes.
7.2.2.2 Position/power results for Group 1, Mr. Singh’s Pain, Part 2. The central position is
shared by facilitator and 1 student, Mary (Figure 62). This student had the highest pretest score
of the group (85%) and therefore it is understandable why she would adopt a central shared
position in the social network and potentially adopt a similar role to the faciltator. The majority
of students are positioned in the peripheral field. Gail have moved from the centre to the
peripheral field, while Sara has moved to midfield (see Table 38).
166
Table 38
Social Network Position/Power Map and Centrality Measures for Group 1, Mr. Singh’s Pain,
Part 2, Build-On Notes
Mr. Singh’s Pain Part 2, build-on notes
Group 1 Participant Eigenvector score Centrality map position
Albert 0.205
periphery
Craig 0.107 periphery
Don 0.087 periphery
Francis 0 n/p
Gail 0.196 periphery
John 0.175 periphery
Jeff 0.176 periphery
Len 0.33 mid
Mary 0.493 core
Sara
0.308 mid
Facilitator 1 0.626 core
Distribution 2 2 6 + 1 n/p
Note. n/p = nonparticipant.
167
Figure 62. Social network position/power map Group 1, Mr. Singh, Part 2, Build-on notes.
7.2.2.3 Position/power results for Group 1, Mary’s Misery. Central position remains shared
by facilitator and the same student, Mary (Figure 63). The majority of students have shifted
position now to the midfield. Albert and Gail have now shifted position from the periphery to
midfield, while Sara remains at midfield (see Table 39).
168
Table 39
Social Network Position/Power Map and Centrality Measures for Group 1, Mary’s Misery,
Build-On Notes
Mary’s Misery, build-on notes
Group 1 Participant Eigenvector Centrality map position
Albert 0.223
mid
Craig 0 n/p
Don 0.111 periphery
Francis 0.083 periphery
Gail 0.239 mid
John 0.157 periphery
Jeff 0.26 mid
Len 0.362 mid
Mary 0.483 core
Sara 0.287
mid
Facilitator 1 0.578 core
Distribution 2 5 3 + 1 n/p
Note. n/p = nonparticipant.
169
Figure 63. Social network position/power map Group 1, Mary’s Misery, Build-on notes.
7.2.2.4 Position/power results for Group 1, Judy’s Last Days, Part 1. Although the central
field remains shared by facilitator and a student, it is now a different student, Albert, who is one
of the highest achievers (90%), as well as one of the largest knowledge gainers (Figure 64).
Mary, formerly at the structural centre has now shifted position to midfield. Students are well
distributed across mid and peripheral fields. “Greatest gainer,” Gail and Sara have not only
shifted position from midfield to the periphery, but with zero scores have not built-on notes (only
read as indicated in the ATK activity analysis). See Table 40.
170
Table 40
Social Network Position/Power Map and Centrality Measures for Group 1, Judy’s Last Days,
Part 1, Build-On Notes
Judy’s Last Days Part 1, build-on notes
Group 1 Participant Eigenvector Centrality map position
Albert 0.5
core
Craig 0.062 periphery
Don 0.062 periphery
Francis 0 n/p
Gail 0 n/p
John 0.25 mid
Jeff 0.275 mid
Len 0.23 mid
Mary 0.399 mid
Sara 0
n/p
Facilitator 1 0.626 core
Distribution 2 4 2 + 3 n/p
Note. n/p = nonparticipant.
171
Figure 64. Social network position/power map Group 1, Judy’s Last Days, Part 1, Build-ons.
7.2.2.5 Position/power results for Group 1, Judy’s Last Days, Part 2. In this last module,
the central field becomes prime real estate (Figure 65). It is shared to an amazing degree by the
facilitator with three students, one of which is Mary, one of the three “incoming stars.” The other
two are new to this central position, Jeff and John. The greatest “gainers” in the group, Albert,
Gail and Sara shifted position from the centre—to the periphery where they have not created any
build-on notes.
It is worth noting that although three students that came into the course with high pretest scores
only one came close to matching the position/power of the facilitator, Mary. The other two did
not. Craig paricipated on the periphery throughout and Len participated in the midfield
throughout the course. It is not known what caused these participants to hold these positions.
172
Half of the participants now occupy the centre in this last module. This can be said to be a shift
toward democracy, where 3 student share the power of ideas and prestige of centrality with the
facilitator. Interestingly the facilitator has retained a strong central position in the problem space.
However, the facilitator shared the central problem space with one or more students in all five
modules (see Table 41).
Table 41
Social Network Position/Power Map and Centrality Measures for Group 1, Judy’s Last Days,
Part 2, Build-On Notes
Judy’s Last Days Part 2, build-on notes
Group 1 Participant Eigenvector Centrality map position
Albert 0.142
periphery
Craig 0.204 mid
Don 0.14 periphery
Francis 0 n/p
Gail 0 n/p
John 0.413 core
Jeff 0.375 core
Len 0.361 mid
Mary 0.406 core
Sara 0
n/p
Facilitator 1 0.56 core
Distribution 4 2 2 + 3 n/p
Note. n/p = nonparticipant
174
7.2.2.6 Position/power analysis interpretation and conclusions for Group 1. The greatest
“gainers” in Group 1, Albert, Gail and Sara shifted position from the core to the periphery across
modules. Albert worked in the periphery of the problem space in the first two modules and them
moved to the midfield in the third module and then to the core and then back to the periphery in
the last module. Although Albert created made few note contributions in the last module , he
read almost all notes (85/96 notes in Module 1; 91/101 in Module 2, and 75/78 in Module 3).
Gail began in the core and then moved to the periphery in the second module and then to the
midfield for the third module. Gail read 95/96 notes in Module 1; 93/101 in Module 2; 70/78 in
Module 3. Sara began in the midfield and remained there for the next two modules and then
moved to the periphery. Sara read 93/96 notes in Module 1; 95/101 in Module 2; 74/78 in
Module 3. Gail and Sara did not participate in the last two modules.
Hence we see a complex relationship of position/power characteristics emerging that defines
greatest “gainer”: (a) depends on incoming level of knowledge, (b) participation for greatest
knowledge gain can be from any position within the problem space, core, mid or periphery, and
(c) gains are evidenced not only by build-on note contributions but also through reading and
understanding others ideas in the problem space. It appears that there is much to be gained from
work at the periphery and midfield, contrary to intuitive understanding and other pedagogic
models such situated learning (Lave & Wenger, 1991).
It is also noteworthy that the core of the Group 1 problem space was always shared by the
facilitator with various students. In Module 1 the core was shared by the facilitator with two
students; in Modules 2, 3, and 4 the core was shared by the facilitator with one student; and in
Module 5 the core was shared by the facilitator and three students. Although the facilitator was
very active, the core was open and in fact demonstrated a strong increase in the final module for
shared opportunities of position and power.
It was hypothesized that the incoming “stars” would most often share the core problem space and
in fact this was found to be true, but only in one of three participation patterns. As expected,
Mary shared position and power with the faciliator in four of five modules. However, Len and
Craig did not participate in the core at all. Len participated in the midfield in four of five
modules and in the periphery; while Craig participated in periphery in four of five modules and
175
in the midfield in the final module. Hence is appears that not all “stars” want to actively
contribute or teach/facilitate.
7.2.2.7 Position/power results for Group 2, Mr. Singh’s Pain, Part 1. The core position in
Mr. Singh’s Pain part 1 is occupied by Facilitator 2 and Shira, one of the highest scorers on pain
pretest (Figure 66). Of particular note is Brenda who achieved the greatest knowledge gain
(28%) and one of the highest posttest scores (85%) in Group 2, who in this module holds a
position in the periphery. Although Brenda did not contribute many build-on notes, she has read
almost all notes (66/69). More than half of the participants occupy a position in the periphery of
the problem space in terms of build-on notes. Like Brenda, Jeorge, Shira, and Sven have all read
almost all notes. Only Phil has read less than 50%. One student, Jack, did not create build-on
notes in this module but read all notes (Table 42). Jenn had the highest incoming score in Group
2 and is positioned in midfield in terms of build-ons notes but has read all notes this first module.
Table 42
Social Network Position/Power Map and Centrality Measures for Group 2, Mr. Singh’s Pain,
Part 1, Build-On Notes
Mr. Singh’s Pain Part 1, build-on notes
Group 2 Participant Eigenvector Centrality map position
Brenda 0.202 periphery
Jack 0 n/p
Jenn 0.353 mid
Jeorge 0.07 periphery
Maria 0.413 mid
Phil 0.045 periphery
Rose 0.014 periphery
Shira 0.509 core
Sven 0.074 periphery
Facilitator 2 0.627 core
Distribution 2 2 5 + 1 n/p
Note. n/p = nonparticipant.
176
Figure 66. Social network position/power map Group 2, Mr. Singh’s Pain, Part 1, Build-ons.
7.2.2.8 Position/power results for Group 2, Mr. Singh’s Pain, Part 2. In Mr. Singh’s Pain
part 2, the core position is shared by the Facilitator 2 and 1 student, Jenn, the incoming star
(Figure 67). Like Mary in Group 1, Jenn entered the course with a relatively high level of
palliative care knowledge, scoring 80% on pretest. Brenda, the greatest gainer in this group has
moved from the periphery to a midfield position. All other students are well distributed across
the mid and pheripheral fields. Jeorge did create build-on notes in this module but read 58/60
notes (Table 43).
177
Table 43
Social Network Position/Power Map and Centrality Measures for Group 2, Mr. Singh’s Pain,
Part 2, Build-On Notes
Mr. Singh’s Pain Part 2, build-on notes
Group 2 Participant Eigenvector Centrality map position
Brenda 0.319 mid
Jack 0.164 periphery
Jenn 0.404 core
Jeorge 0 n/p
Maria 0.37 mid
Phil 0.056 periphery
Rose 0.235 mid
Shira 0.268 mid
Sven
0.288 mid
Facilitator 2 0.598 core
Distribution 2 5 2 + 1 n/p
Note. n/p = nonparticipant.
178
Figure 67. Social network position/power map Group 2, Mr. Singh’s Pain, Part 2, Build-ons.
7.2.2.9 Position/power results for Group 2, Mary’s Misery. Central field position is held by
Facilitator 2 and not shared by any students in this module (Figure 68). Jenn has shifted position
from the core to the midfield. Brenda remains in midfield. Others are distributed across midfield
with only one student in the peripheral field. Jack and Jeorge did not create build-on notes in this
module but read all notes. Phil did not create any build-on notes and read less than 50% of the
notes. See Table 41.
179
Table 44
Social Network Position/Power Map and Centrality Measures for Group 2, Mary’s Misery,
Build-On Notes
Mary’s Misery, build-on notes
Group 2 Participant Eigenvector Centrality map position
Brenda 0.404 mid
Jack 0 n/p
Jenn 0.312 mid
Jeorge 0 n/p
Maria 0.356 mid
Phil 0 n/a
Rose 0.058 periphery
Shira 0.215 mid
Sven 0.26 mid
Facilitator 2 0.704 core
Distribution 1 5 1 +3 n/p
Note. n/p = nonparticipant.
180
Figure 68. Social network position/power map Group 2, Mary’s Misery, Build-on notes.
7.2.2.10 Position/power results for Group 2, Judy’s Last Days, Part 1. The structural core is
shared by Facilitator 2 and Jenn once again. Jenn shifted position from the midfield to the core in
this module (Figure 69). Brenda shifted position from midfield to the periphery. Although
Brenda and Jack are in peripheral positions, Brenda read 35/36 notes and Jack read all. Five of
the 9 students did not create build-on notes in this module. Juan and Phil read less than 10% of
notes; Rose read less than 50%; while Shira read all notes and Sven read 33/36. See Table 45.
181
Table 45
Social Network Position/Power Map and Centrality Measures for Group 2, Judy’s Last Days,
Part 1, Build-On Notes
Judy’s Last Days Part 1, build-on notes
Group 2 Participant Eigenvector Centrality map position
Brenda 0.119 periphery
Jack 0.114 periphery
Jenn 0.631 core
Jeorge 0 n/p
Maria 0.32 mid
Phil 0 n/p
Rose 0 n/p
Shira 0 n/p
Sven 0 n/p
Facilitator 2 0.687 core
Distribution 2 1 2 + 5 n/p
Note. n/p = nonparticipant.
182
Figure 69. Social network position/power map Group 2, Judy’s Last Days, Part 1, Build-ons.
7.2.2.11 Position/power results for Group 2, Judy’s Last Days, Part 2. The core position in
Judy’s Last Days part 2 is shared by the Facilitator 2 and a student that has not been at the core
before, Maria (Figure 70). Jenn shifted position from the core to midfield. Brenda shifted
position from the periphery to midfield. The same five students, as in the previous module, did
note create build-on notes in this module. See Table 46.
183
Table 46
Social Network Position/Power Map and Centrality Measures for Group 2, Judy’s Last Days,
Part 2, Build-On Notes
Judy’s Last Days Part 2, build-on notes
Group 2 Participant Eigenvector Centrality map position
Brenda 0.359 mid
Jack 0 n/p
Jenn 0.403 mid
Jeorge 0 n/p
Maria 0.506 core
Phil 0 n/p
Rose 0 n/p
Shira 0 n/p
Sven 0 n/p
Facilitator 2 0.673 core
Distribution 2 2 6 n/p
Note. n/p = nonparticipant.
184
Figure 70. Social network position/power map Group 2, Judy’s Last Days, Part 2, Build-ons.
It is noteworthy that various students have taken on position at the centre. Moveover, the
position at the centre was never occupied by the student making the greatest knowledge gains,
student Brenda. This student participated in the mid and peripheral fields only. Jenn participated
the most often at the structural core of the social network, like Mary in Group 1, Jenn came into
the program with a high level of knowledge as evidenced by her pretest score. Therefore the
pattern appears across groups, indicating that some (but not all) students with high levels of
knowledge in the subject area will share position and power with the facilitator. Overall
positional analysis in both Groups 1 and 2 are well distributed. The dynamic and changing nature
of students positions within a social network over a long period of time should be noted (1 month
per module, across five online case-based modules). The persistent central position of
185
Facilitators 1 and 2 denotes their communal engagement in the Knowledge Building process.
However, this structural analysis of position and power does not inform us as to how the
facilitators engage with participates. Can the facilitator/student structure be characterized to
determine what kind of stance is most productive in scaffolding Knowledge Building, that of
monitor, mentor, and/or partner as described by Tabak and Baumgartner (2004).
7.2.2.12 Position/power analysis interpretation and conclusions for Group 2. Centrality
mapping and power analysis demonstrated facilitator/student shared power spaces in four of the
five Group 2 modules. Facilitator 2 remained at the core in all modules. Students with the
greatest incoming knowledge test scores were the ones that shared the central power space with
facilitators initially. Advances in knowledge gains were demonstrated by those that participated
at the periphery in terms of build-on notes; many of these gains were supported by exemplary
reading scores of notes read. This finding supports Zhang and colleagues’ (2002) study that
focused on the importance of strong reading scores to support Knowledge Building.
In conclusion, students with high levels of incoming knowledge may tend toward sharing the
core problem space and role of facilitation—but not always. Position and power may change
over time. Peripheral positions in a number of cases, in this study, led to greatest knowledge
advances. We are therefore in need of reconceptualizing commonplace associations of position
and power, and our notions of what constitutes participation. Apparently there is much to be
gained from working at the periphery and sometimes this work is through note contributions but
other times it is through high levels of reading of others ideas. Democratization of Knowledge
Building was demonstrated through a stunning shift is position by students to the core in the final
module in Group 1 and continuous sharing of core position and power with participants in
Group1; shared position and power was also evident in Group 2 but not across all modules and
not to the same degree.
Further research is required to understand the complexities of social network power
relationships, beyond structural findings of centrality position, involving how ideas-at-the-centre
of a problem space are worked with, in terms of the power of an idea, participant previous
knowledge, and personal preferences for knowledge work. These sociocognitive relationships are
further expressed in the results of within notes analyses of discourse. Content analyses results
will be presented next.
186
7.3 Results of Social Network Content Analyses
This section presents results of within note content analysis to further advance explanations and
understandings of previous social network structural and position/power results. Content
analyses results herein relates specifically to the details stated in the second part of my 6th
research subquestion: What are the social network structural and content relationships between
position/power and knowledge improvement; and are these relationships reflected in
facilitator/student characteristics, complexity of discourse, themes of belief and design-mode
work, and the demonstration of indicators of Knowledge Building?
Results of the following series of analyses are presented below:
· Relationships between facilitator and students, demonstrated through facilitator
stance and comparison of facilitator/student patterns of statements and questions;
· Thematic results of content: emergent ideas, above and beyond learning objectives
and relationship to adductive and abductive knowledge work;
· Complexity of discourse using Bordage scale for Semantic Analysis of Clinical
Discourse (1991); and,
· Results of analysis of Knowledge Building indicators and exemplars.
Based on the results of the previous section on centrality position/power analysis four modules
were selected, two modules from Group 1 and two modules from Group 2. Mr. Singh’s Pain,
Part 2 was selected based on the fact that these modules in both groups were represented by a
student participant and facilitator at the core and well distributed networks. In addition, this
module was one month into the program, so participants were well acquainted with KF technical
functions and social engagement online. In terms of content, this module deals with some key
principles reflected in items on the pain knowledge and beliefs pre-/posttests.
The second modules selected for further analyses across both groups was the final module,
Judy’s Last Days, Part 2. This module were specifically selected for within note examination to
help provide further understanding of the strong shift in position by numerous Group 1 students
to the core.
187
7.3.1 Social Network Analysis of Facilitator/Students Patterns of Discourse
Patterns of facilitator/student interactions based on within note discourse analyses of Groups 1
and 2 across 40% of the dataset— specifically all notes in Mr. Singh’s Pain, Part 2 and Judy’s
Last Days, Part 2—were scored by the researcher. Two of the four modules were scored by a
second rater. Lack of clear definition of categories led moderate interrater agreement (Mr. Singh,
Group 1 Kappa = 0.446 with p < 0.000; Group 2 Kappa = 0.566 with p < 0.000). Tabak and
Baumgartner’s (2004) categories of monitor, mentor and partner were expanded to include the
facilitator stance of expert, characterized by didactic teaching; in-between categories of
mentor/partner and partner/expert were also added. The results are presented below.
7.3.1.1 Patterns of facilitator discourse. One of the novel findings in this study is the high
level of facilitator participation in the discourse. Although we have determined that facilitators
were central to the social network, we do not know how they participated and what
sociocognitive facilitator/student dynamics resulted. Results of analysis of patterns of
facilitator/participant discourse patterns, characterized as facilitator stance, are indicated in Table
47. A series of characteristics defined the stance of discourse and was used in analysis and
scoring.
Table 47
Facilitator/Participant Discourse Patterns: Discourse Stance
Facilitator/Participant Discourse Patterns
Monitor Mentor Mentor/Partner Partner Partner/Expert Clinical
expert
Group 1, Facilitator 1
Mr. Singh’s Pain, Pt.2
(n of notes = 31/37)
3 7 1 8 8 4
Judy’s Last Days, Pt. 2
(n of notes = 20)
2 3 0 5 7 3
Group 2, Facilitator 2
Mr. Singh’s Pain, Pt. 2
(n of notes = 26)
7 4 0 4 6 5
Judy’s Last Days, Pt. 2
(n of notes = 13)
1 2.5 1 2 1 5.5
Totals 13 16.5 2 19 22 17
188
Results of pattern counts in Group 1 across the two modules examined are strongest in the
categories of facilitator as mentor, partner, and partner/expert. In Group 2 pattern demonstrates
highest scores in categories of facilitator as monitor, and as partner/expert and clinical expert.
Collectively all facilitator roles are acknowledged. This demonstrates the necessary flexibility of
skills required by facilitators and the range of roles necessary to support online work. These
patterns are both indicative of facilitator/student symmetry and asymmetry discourse patterns
and shifts between contributory and authoritative stance. This is a significant departure from
traditional stance of instructor as clinical expert in CME. Although this role is acknowledged in
online collaborative knowledge work, it is not the dominant role in three of the four modules.
Facilitator 2 uses this role predominantly in the last module to do much didactic teaching.
Facilitation of the two Group 1 modules are strongly characterized by more democratic discourse
enabled by greater use of more symmetric roles and less use of the asymmetric, didactic role of
the clinical expert.
Highest counts for Facilitator 1 were in categories of partner and partner/expert that is
characterized as inquiry in design and belief modes. Highest count for Facilitator 2 was the role
of monitor, which is characterized by administrative inquiry. Lower count for monitoring by
Facilitator 1 is likely due to differences is student characteristics; that is, Group 1 participants
demonstrated more agency and therefore needed less monitoring to participate than students in
Group 2.
Tabak and Baumgartner (2004) found the role of teacher-as-partner important to identity
formation and inquiry-based science learning in classrooms with young students. In this study
facilitator as partner and expert emerged as strong roles supporting online inquiry and knowledge
work. Further examination of discourse patterns is required to understand the moves and the
process of knowledge improvement. Thus, patterns of discourse statements and questions are
analyzed in the next section.
7.3.1.2 Patterns of discourse statements and questions. Didactic teaching is often
characterized as knowledge telling—a one-way process conveying information through a series
of statements. This next analysis was driven by the hypothesis that questions, on their own or
along with statements, drive Knowledge Building. It can be said that the relationship between
questions and statements scaffold knowledge growth and drive abductive and adductive
knowledge work, more broadly and deeply. Therefore, in this next analysis, evidence of
189
questions (through numeric count) was established, along with the relationship between
questions and statements. Notes were categorized by the researcher as:
· Statement followed by a question;
· Question only;
· Question followed by a statement;
· Statement only; or
· Administrative/other
In addition, students-at-the-core and the facilitator at-the-core (based on previous social network
position/power analysis) were examined to compare question/statement patterns (Table 48).
Table 48
Patterns of Discourse Statements and Questions by Facilitators and Students,
and Student/Facilitator Patterns at the Core
Note. S = Statement. Q = Question. Total = Question Only + Statement followed by Question + Question followed
by Statement. Facilitator-at-the-core + All Students = ALL.
S followed
by Q
Q only
Q followed
by S
Total Q
+ Q/S
S only
Admin.
Mr. Singh’s Pain, Part 2
Group 1
Student-at-the-core (n = 1) 7 0 0 7 11 0
Facilitator-at-the-core (n = 1) 5 6 2 13 14 6
All Students 18 5 0 25 37 3
ALL 23 11 2 36 51 9
Group 2
Student-at-the-core (n = 1) 1 0 0 1 4 1
Facilitator-at-the-core (n = 1) 3 1 2 6 12 8
All Students 7 2 0 9 23 0
ALL 10 3 2 15 35 8
Judy’s Last Days, Part 2
Group 1
Student-at-the-core (n = 3)* 4 0 0 4 17 0
Facilitator-at-the-core (n = 1) 5 1 2 8 7 4
All Students 8 0 1 9 27 2
ALL 13 1 3 17 34 6
Group 2
Student-at-the-core (n = 1) 2 0 0 2 1 1
Facilitator-at-the-core (n = 1) 3 0 1 4 8 1
All Students 5 0 0 5 7 3
ALL 8 0 1 9 15 4
190
Results of this count and discourse pattern analysis indicate that students at the core of the social
network most frequently made statements or made statements followed by a question(s). In the
modules examined here in there were no instances of students at the core posting a note in KF
with only a discourse question or a question followed by a statement. Similarly, facilitators at the
core most frequently made statements and made statements followed by a question within the
same note. However, facilitators also demonstrated other discourse patterns of posting questions
only and questions followed by statement(s) in a KF note.
Group 1 students and facilitator at the core posted more questions that those in Group 2. In Mr.
Singh’s Pain, Part 2, the facilitator for Group 1 posted six questions while the facilitator for
Group 2 posted one question. In the same modules the Group 1 facilitator cumulatively posted 13
questions in all categories involving questions while the Group 2 facilitator only posted six.
Similar discourse patterns were found for all students. In all modules examined students posted
statements most frequently. In three of the four modules students posted notes with statements
followed by a question(s) approximately 30% to 50% as often as notes with statements only.
These discourse patterns are striking in that collective knowledge work proceeds not by
knowledge telling through statements alone but by inclusions of questions with statements or by
questions alone. Discourse patterns including questions are highly prevalent, particularly
statements followed by questions by both facilitators and students. Thus, we can clearly see that
student and facilitator questions play an important role in advancing knowledge work and can be
considered a demonstration of intentional Knowledge Building and epistemic agency to improve
ideas. However we do not know how these questions and statements are related to the interaction
of ideas—to work in belief-mode and design-mode. Some questions are intended only to clarify
facts, like how to titrate dosages of pain medication, and are based on the predetermined learning
objectives posted in the module. Other kinds of questions are defined as those linked to new and
emergent ideas intended to drive Knowledge Building deeper, adductive Knowledge Building or
expand ideas more broadly, abductive Knowledge Building.
The next analysis was aimed at determining whether participant discourse and ideas remained
within the constraints of the predetermined learning objectives in each case-module or if
emergent ideas above and beyond the predetermined learning objectives were present in the
discourse, and if so, what are the emergent themes identified.
191
7.3.2 Content Analysis of Themes, Beyond the Predetermined Learning Objectives
7.3.2.1 Content analysis of knowledge work. This next set of analyses by the researcher
employs the same four modules used previously for within-content analysis. The focus of these
analyses was to determine if within note discourse patterns demonstrated belief-mode and
design-mode knowledge work through emergent ideas. The objective was to obtain an
understanding of the relationship between predetermined knowledge objectives (set forth by the
course content designers), in terms of how participants in the problem space worked with these
objectives. The question was: Is there evidence of knowledge work above and beyond these
objectives, that could be distinguished from the learning objectives and identified as an emergent
idea and/or theme, consistent with the Knowledge Building notion of design-mode and belief-
mode work? Within-notes analysis of the problem space was conducted. Social network
knowledge work by students and facilitators were examined.
Results indicated that less than 49% and 48% of knowledge work in Group 1 across both
modules related to predetermined learning objectives and 51% and 52% of the discourse was
around emergent ideas, beyond those listed in the learning objectives (Table 49).
In contrast, the majority of Group 2 notes were knowledge work related to the preidentified
learning objectives. In Mr. Singh’s Pain, Part 2, 57.63% of knowledge work with notes pertained
to the learning objectives. Similarly in the last module, Judy’s Last Days, Part 2, 73.33% of
participants worked with the predefined learning objectives. Only 41.67% and 33.33% of
participants in the problem space identified emergent issues for knowledge work (Table 49).
Table 49
Results of Knowledge Work With Predefined Learning Objectives or Emergent Ideas
Total n of notes n of notes related to
objectives (%)
n of notes beyond
objectives/emergent ideas (%)
Group 1
Mr. Singh’s Pain, Part 2 106 52 (49.07) 54 (50.94)
Judy’s Last Days, Part 2 64 31 (48.44) 33 (51.56)
Group 2
Mr. Singh’s Pain, Part 2 59 34 (57.63) 25 (41.67)
Judy’s Last Days, Part 2 30 22 (73.33) 8 (26.67)
Note. n = number.
192
Emergent ideas beyond learning objectives were identified across both groups and in all
modules. Work with ideas defined by the learning objectives and work with emergent ideas
beyond the learning objectives was very balanced by Group 1. Thus, we can conclude that in this
study, knowledge work proceeds in belief and design-mode and that there is a relationship
between these modes of work. Knowledge work of this nature seems to involve abductive,
broadening of ideas, or adductive, deepening ideas to improve understanding and may help to
explain why Group 1 experiences greater knowledge gains. In the next analysis themes were
identified through the categorization of discourse threads.
7.3.2.2 Relationship of themes, threads, and learning objectives in the social network
discourse. Knowledge Building through belief-mode and design-mode or adduction and
abduction of knowledge and ideas can be connected to content themes through categorization of
threads (Zhang, et.al, 2009). Themes, in turn can be identified as emergent or related to
predetermined learning objectives. The results of this analysis are presented below and
summarized in Table 50.
Themes identified in discourse threads related to knowledge work based on the predefined
objectives in Mr. Singh’s Pain, Part 2, which were:
· Pain management
· Opioid use
· Neuropathic pain
· Bone pain
Knowledge work in belief and design-mode work in Mr. Singh’s Pain, Part 2, was characterized
by adductive and abductive knowledge work with new ideas. Nine new themes emerged in the
Group 1 discourse and were identified as:
· Personal cases
· Authentic practice
· Cost of drugs
· In-practice or “how-to” details
· Local access to palliative nursing care and support services
· Dealing with personal emotions
· Communication with patients and family members
193
· Better understanding of various cultural and religious practices
· Self-care—how to care for oneself as a palliative care physician
Participants identified many important ideas for discourse and as indicated above. Authentic new
themes emerged beyond the predetermined learning objectives. Analogies from the case of Mr.
Singh were made to personal cases from physicians’ own practices and authentic issues from
their own experiences. In Group 1, 4 students provided personal cases to broaden the problem
space.
In Group 2, one student contributed a personal case from their own practice to the problem
space. This type of emergent contribution shifts the discourse and enables students to become co-
designers of the curriculum, metadesigners of the problem space. Thematic topics, content, and
ideas of importance to them are thus virtually added to the predetermined objectives defined by
the curriculum designers. Some themes identified by Group 2 were the same as those in Group 1;
others were not—new themes have been italicized below. Group 2 themes include:
· Personal cases
· Authentic practice
· Local access to palliative care support (e.g., community nurses)
· Emotions
· Self-care
· Cultural practices
· Communication with patients and families
· Education of families
· Coverage
· Funerals
Analogies to personal experiences are common throughout this discourse. The emergence of
discourse around how treating physicians can deal with emotions and whether or not to attend a
patient’s funeral are important professional practice ideas that are well beyond the learning
objectives originally identified by curriculum designers for this module—but perhaps the most
important. These are themes and ideas rarely found in typical continuing medical education
course and clinical references on palliative care.
194
Themes identified in discourse threads related to predefined objectives and knowledge work in
Judy’s Last Days, Part 2 were:
· Management of last hours of life
· Family concerns and counseling
· Signs of imminent death
· Symptoms
Similar to the previous modules examined, discourse in Judy’s Last Days, Part 2, clearly
demonstrated numerous emergent issues and ideas, categorized as themes and threads of beyond
the learning objectives.
Table 50
Summary of Emergent Themes/Threads Beyond Learning Objectives
Mr. Singh’s Pain, Part 2
Learning objectives Emergent themes/threads in abductive and adductive KB beyond objectives
1. Pain management
2. Opioid use
3. Neuropathic pain
4. Bone pain
Group 1
(N of notes =
106)
Authentic practice
Group 2
(N of notes =
59)
Authentic practice
Personal cases Personal cases
Drug cost Support/Emotions
In-practice Funerals
Local access Local Access
Emotions Coverage
Communication Communication
Culture/Religion Culture
Self-care Educate Family
Self-Care
1. Management of last
hours of life
2. Family concerns and
counseling;
3. Signs of imminent
death
4. Symptoms
Group 1
(Number of
notes = 64)
Authentic practice
Group 2
(Number of
notes = 30)
Authentic practice
Personal cases Personal cases
Culture/Religion Funerals
Teamwork Culture/Religion
Homecare Home visits
Local Access Death certificate
Suffering Billing/OHIP codes
Communication
Death certificate
Self-care
Emotions
In conclusion, codesign of the curriculum becomes deeper and broader through student
contributions of analogous cases, authentic experiences, and identification of personal
knowledge gaps. Group 1 identified 11 themes beyond the predetermined objectives; some were
similar to those from Mr. Singh’s Pain, Part 2, but intentionally further broadened ideas of
195
discourse. Again personal cases from individual physicians’ private practices were introduced to
into the problem space. Philosophic and clinical discussions around the concept of “suffering”
emerged as a central issue. Group 2 identified 7 emergent themes; some overlapped those
identified by Group 1 and also included student contributions of their personal practice cases.
Both groups identified professional practice gaps in knowledge around how to issue a death
certificate, billings and OHIP codes.
7.3.2.3 Summary of emergent themes and metadesign results. It is clear from the foregoing
analysis that numerous important themes, beyond the predefined learning objectives for these
modules, emerge in the discourse. Emergent themes are different according to Groups, and the
students in those groups and the facilitator that facilitates them.
Student level of knowledge, lack of knowledge, interests, and other things that usually have no
place in a curriculum, like physician emotions around death and dying, become artifacts of
discourse within the problem space for design-mode and belief-mode work in Knowledge
Forum. Facilitator centrality in the social network, is flexible, demonstrated by shifting roles of
mentor, partner, and expert, enabling corresponding student shifts toward democracy in the
discourse. Participant clinical discourse was found to be at a very high level, characterized as
elaborated and compiled.
It was posited that questions drive Knowledge Building and results indicated that questions are
contributed to the discourse by students, particularly those at the core of the social network;
questions are not the sole purvey of the facilitator. The foregoing analyses demonstrated strong
social network content patterns and a positive relationship between design-mode knowledge
work and new ideas that I have called abductive and adductive knowledge work. Abduction
intentionally broadens knowledge by explaining more phenomena, for example in this study
students contributed analogous cases to the discourse space to broaden understanding of patients
symptoms during the last days of life. Working more and more deeply with facts can be thought
of as both adductive and abductive Knowledge Building. Adduction can be defined as the
intentional deepening of understanding by investigation of deeper layers and/or greater details.
An example of abductive and adductive knowledge work from the discourse is clarification of
titration of opioids and discussion around the nuances of opioid treatment as the “art and
science” of pain management.
196
Strong evidence in this study supports and elucidates the complex aspects of Knowledge
Building. Online design of this continuing medical education program in Knowledge Forum was
both structured/prescribed by expert-driven objectives and intentionally unstructured for
emergent knowledge work to be determined by participants, students and facilitator, to enable
both belief-mode and design-mode Knowledge Building.
Thus, it can be said that students in this continuing medical education course have had the
opportunity to participate in the collaborative design of their knowledge work and to imbue the
problem space with their concerns, authentic problems, and real-world practice issues, as well as
highly personal and ethical facets, not typically brought into the discourse of traditional medical
teaching environments. In essence they have worked to advance community knowledge and
through their emergent ideas they have also contributed to the metadesign of the curriculum. I
have termed this type of Knowledge Building metadesign democratic innovation for continuing
medical education. Democratic innovation includes not only sociocognitive participation, but
emergent codesign of curriculum—work typically assumed by teachers and experts, not by
participants.
7.3.3 Social Network Analysis of Complexity of Discourse
The results below delve into the complexity of discourse and are categorized according to the
Bordage scale of complexity of clinical discourse (1991). The same four modules were used for
analysis from Group 1 and 2, Mr. Singh’s Pain, Part 2 and Judy’s Last Days, Part 2. This
represents 40 % of the dataset which was scored by the principal investigator; three of the four
modules were scored by a second rater. Interrater reliability was calculated using Cohen’s Kappa
coefficient. Interrater agreement was moderate (Mr Singh, Group 1, Kappa = 0.474 with p <
0.000; Mr Singh, Group 2, Kappa = 0.504 with p < 0.000; Judy’s Last Days, Group 2, Kappa =
0.427 with p < 0.000). Categories of elaboration and compilation may have been too similarly
defined.
Bordage’s scale of semantic analysis of clinical discourse (Bordage et al., 1991) was used to rate
complexity of discourse within notes. The 4-point scale categorizes complexity of clinical
discourse as reduced, dispersed, elaborated or compiled. These terms are defined as follows:
· Reduced discourses, typically characterized by little exposition of thought, often
limited or inaccurate knowledge.
197
· Dispersed discourses, typically characterized by numerous often unrelated diagnostic
possibilities, both reasonable and unreasonable; thought processes are not organized.
· Elaborated discourses, characterized by the use of SQs (semantic qualifiers) to
compare and contrast diagnoses, either explicitly or at times, implicitly; the
clinician’s train of thought is clear.
· Compiled discourses, characterized by structured, concise, and condensed syntheses
of case data using selected SQs (semantic qualifiers) to clearly “rule in” and “rule
out” pertinent diagnostic possibilities; most often found in the discourses of
experienced, expert clinicians. (Bordage et al., 1991, p. 3).
Case and online library notes created by the curriculum design team were not scored or included
in this analysis. There were no clinical discourse notes rated as reduced or dispersed. However,
there were some administrative notes that were categorized and placed these categories.
Examples of reduced/dispersed administrative discourse are notes containing notification of
online tests, reminders of dates for next modules, communication about participant holidays, etc.
In the last module, Judy’s Last Days, Part 2, “other” types of notes emerged toward the end of
the course; these were thank you notes, reflections on the course and on practice. These “other”
notes were categorized as reduced/dispersed and put together with the administrative notes.
Social network analysis of complexity of discourse was similar across all modules in that almost
all student and facilitator discourse was rated as complex or elaborated. Since all participants in
this CME study are family physicians, it is not surprising that all clinical discourse reflects a high
level of complexity, within notes, across all four modules scored (Table 51).
It is noteworthy that facilitator elaborated discourse notes are more prominent than complied in
three of the four modules. Group 1 Facilitator wrote approximately 30% more elaborated notes
than compiled. Group 2 Facilitator wrote approximately 30% less elaborated notes than compiled
notes in Mr. Singh’s Pain, Part 1; and more than 50% more elaborated notes than compiled notes
in Judy’s Last Days, Part 2.
In contrast, student note complexity, in three of the four modules, is almost evenly distributed
between compiled and elaborated discourse. Only students in Group 1, in Judy’s Last Days, Part
2, exhibited 15.63% compiled discourse and 40.63 elaborated discourse complexity. However, it
can be said that both students and facilitators demonstrated a high level of Knowledge Building
198
work that was well -distributed and demonstrated agency to compile and synthesize ideas, as
well as to explore and elaborate their understandings.
Table 51
Semantic Analysis Results of Complexity of Discourse
2008/2009 Complexity of discourse results
Group 1 Mr. Singh’s Pain, Part 2
(n of notes = 106)
Judy’s Last Days, Part 2
(n of notes = 64)
Compiled
% (n)
Elaborated
% (n)
Admin. (n) Compiled
% (n)
Elaborated
% (n)
Admin./
Other (n)
Students
(n = 10)
30.19
(32)
32.08
(34)
2.83
(3)
15.63
(10)
40.63
(26)
12.5
(8)
Facilitator
(n = 1)
11.32
(12)
17.92
(19)
5.66
(6)
10.94
(7)
14.06
(9)
6.24
(4)
ALL 41.51
(44)
50
(53)
8.49
(9)
26.56
(17)
54.69
(35)
18.75
(12)
Group 2 Mr. Singh’s Pain, Part 2
(n of notes = 59)
Judy’s Last Days, Part 2
(n of notes = 30)
Compiled
% (n)
Elaborated
% (n)
Admin. (n) Compiled
% (n)
Elaborated
% (n)
Admin./
Other (n)
Students
(n = 9)
27.12
(16)
27.12
(16)
1.69
(1)
20
(6)
20
(6)
13.33
(4)
Facilitator
(n = 1)
16.64
(11)
11.86
(7)
13.56
(8)
10
(3)
26.66
(8)
6.66
(2)
ALL 45.76
(27)
38.98
(23)
15.25
(9)
30
(9)
46.66
(14)
23.33
(7)
Although we now know that the discourse was complex and emergent and students worked
intentionally to improve knowledge in belief and design mode, we have not verified that
principles of Knowledge Building were evident in the discourse. Results of this analysis will be
provided next.
7.3.4 Evidence of Knowledge Building Indicators Within Social Network Discourse
The final facet of results to be presented is the representation of all Knowledge Building
principles within the discourse of these four modules. The same four modules were scored on a
5-point scale (Sibbald, 2009) indicating degree to which each module demonstrated each of the
12 Knowledge Building principles as defined by Scardamalia (2002). The principle of pervasive
Knowledge Building was scored only in relationship to the context of this program since it is not
199
known if Knowledge Building occurred beyond this program in other aspects of participants’
work and lives. The four modules were scored by the researcher and a second rater. Interrater
reliability was high (83%). The two raters agreed 10 out of 12 times.
All Knowledge-Building principles were evidenced in each module scored; many were found to
be strong and consistent, as indicated in Table 52. Principles of community responsibility,
democratic knowledge, idea improvement, diversity, Knowledge-Building discourse, and work
with real ideas/authentic problems were prominent. Indicators of epistemic agency and rise-
aboves rated more cautiously to acknowledge broader context of ideas beyond the local use.
Themes of emergent ideas were clearly over and above learning objectives, and clearly
demonstrated metadesign knowledge work.
Table 52
Knowledge Building Principles Demonstrated in the Discourse
Knowledge Building Principle
Strong and
consistent
Relatively
strong and
consistent
Observed but
inconsistent
Occasional
Not
observed
Real ideas, authentic problems x
Knowledge Building discussion x
Improvement of ideas x
Diversity of ideas x
Conceptual change:
understanding that rises above
previous ideas
x
Epistemic agency x
Community knowledge/
Collective responsibility x
Democratizing knowledge x
Symmetric knowledge
advancement x
Constructive use of authoritative
sources x
Embedded transformative
assessment (through discourse) x
Pervasive Knowledge Building x
200
Group 1 in particular provided numerous instance of emergent ideas. As previously
demonstrated, emergent ideas formed nearly 50% of the discourse and provided clear
demonstration of design-mode work and metadesign of the curriculum. A series of notes from
the 2008/2009 discourse are provided below as evidence and exemplars of Knowledge Building
principles of community knowledge, collective responsibility, working to improve ideas,
democratic and emergent knowledge work.
The first example (Figures 71, 72, 73) is a series of three discourse notes from Mr. Singh’s Pain,
Part 2. In the first note the student identifies a knowledge gap, her lack of understanding around
a complex issue that may not have a “right/wrong” answer. Epistemic agency is demonstrated
through her identification of the issue and her attempt to find the answer to this question by
reading various references; still fuzzy, the student posts questions to the problem space. The
student exemplifies “work at the edge of knowledge” (Scardamalia, 2003) demonstrating the
Knowledge Building principle of “constructive use of authoritative sources” both through
accessing written references and the KF problem space, as well as internal assessment,
identifying an important problem of interest to the community, exemplifying the principle of
“embedded and transformative assessment.” The facilitator responds didactically and then adds
insight based on her own professional practice. The third note, by another student, recognizes the
initial student’s epistemic efforts. In many ways, this interchange is characteristic of “rise-above”
Knowledge Building, work with higher-level formation of problems that involves complex and
messy ideas, but result in moving to a higher plan of understanding (Scardamalia, 2002).
Figure 71. Exemplar 1: 2008/09, Group 1, Mr. Singh’s Pain, Part 1.
201
Figure 72. Exemplar 2: 2008/09, Group 1, Mr. Singh’s Pain, Part 1.
Figure 73. Exemplar 3: 2008/09, Group 1, Mr. Singh’s Pain, Part 1.
202
In the next example (Figures 74–85), the interchange begins with a series of questions from the
facilitator. The build-on responses are numerous and include two personal practice case
analogies that extend the discourse. The flow of the discourse evolves into spirituality issues,
advanced directives, practical issues of how to pronounce and fill out a death certificate,
logistical issues, questions about attending patient funerals, and reflections on “Why we do it.”
This powerful discourse is triggered by the facilitator and then taken over by the students—one
student emerges as the leader and takes over the role of facilitator; this is one of the students
positioned at the core. This interaction provides strong exemplars of numerous Knowledge
Building principles, including democratizing knowledge, real ideas/authentic problems,
improvable ideas, diversity, epistemic agency, rise-above, community knowledge/collective
responsibility, symmetric knowledge advancement, involving both belief-mode to advance the
edge/deepen understanding and design-mode work to broaden knowledge and experience.
Figure 74. Exemplar 4: Group 1, Judy’s Last Days, Part 2.
204
Figure 76. Exemplar 6: Group 1, Judy’s Last Days, Part 2.
Figure 77. Exemplar 7: Group 1, Judy’s Last Days, Part 2.
205
Figure 78. Exemplar 8: Group 1, Judy’s Last Days, Part 2.
Figure 79. Exemplar 9: Group 1, Judy’s Last Days, Part 2.
206
Figure 80. Exemplar 10: Group 1, Judy’s Last Days, Part 2.
Figure 81. Exemplar 11: Group 1, Judy’s Last Days, Part 2.
207
Figure 82. Exemplar 12: Group 1, Judy’s Last Days, Part 2.
New Note:
Figure 83. Exemplar 13: Group 1, Judy’s Last Days, Part 2.
208
Figure 84. Exemplar 14: Group 1, Judy’s Last Days, Part 2.
Figure 85. Exemplar 15: Group 1, Judy’s Last Days, Part 2.
209
At the conclusion of the last module, Group 1 students choose to voice their opinions about the
course and reflections on the process (Figures 86–89). The strength of the Knowledge Building
community and appreciation for opportunistic collaboration and metadesign rings clear in the
notes below.
Figure 86. Exemplar 16: Group 1, Judy’s Last Days, Part 2—Reflections
Figure 87. Exemplar 17: Group 1, Judy’s Last Days, Part 2—Reflections
210
Figure 88. Exemplar 18: Group 1, Judy’s Last Days, Part 2—Reflections
Figure 89. Exemplar 19: Group 1, Judy’s Last Days, Part 2—Reflections
7.3.5 Cluster 3 Summary of Social Network Position and Power Analysis, Sociocognitive
Dynamics, and Indicators of Knowledge Building
This cluster was guided by the investigations of the following research subquestion: What are the
social network relationships between structural position, power (defined as centrality of ideas),
and knowledge improvement; and how are these relationships reflected in facilitator/student
sociocognitive dynamics of Knowledge Building (beyond learning objectives), through emergent
themes, in complexity of discourse, and by indicators of Knowledge Building?
211
7.3.5.1 Sociocognitive dynamics of social network position/power.
7.3.5.1.1 Facilitator/students patterns of discourse patterns. Strongest Group 1 Facilitator
discourse pattern is as partner and partner/expert; strongest Group 2 Facilitator discourse pattern
is as monitor and partner/expert. In regards to statement/questions patterns, students and
facilitators at the core of the social network most frequently made statements or made statements
followed by a question(s). Overall, student/facilitator discourse patterns are striking in that
collective knowledge work proceeds not by interchange of statements alone; but most often
through statements followed by a question(s). Group 1 Facilitator posed more than 50% more
questions in both modules examined, than Group 2 Facilitator.
7.3.5.1.2 Results of analysis of complexity of discourse. Social network analysis of
complexity of clinical discourse was similar across all modules in that all student and facilitator
discourse was rated as complex or elaborated. No discourse was rated as dispersed or reduced.
Since all participants in this study are family physicians, it is not surprising that all clinical
discourse reflects a high level of complexity, within notes, across all four modules scored.
7.3.5.1.3 Emergent themes and metadesign knowledge work (beyond predefined learning
objectives). Emergent themes were identified in Group 1 and Group 2 discourse. Emergent
knowledge work can be characterized as adductive and abductive Knowledge Building, working
to improve knowledge so that it is understood more deeply and broadly. Group 1 and Group 2
identified different themes for knowledge work according to their needs and interests. Analogous
cases, “my experience,” “in my practice,” emotions, and pragmatics of practice (e.g., OHIP
billings). Personal knowledge lacks were identified, “need to know” questions were posted,
misconceptions corrected, and grey areas discussed that have no one answer (e.g., the art and the
science of palliative care practice).
7.3.5.1.4 Results of Knowledge Building indicators within social network discourse. All
Knowledge Building principles were evidenced in each module scored and most were found to
be “strong and consistent.” Exemplars of principles of community responsibility, democracy,
improvable ideas, epistemic agency demonstrated primarily through emergent themes/ideas, are
indicative of metadesign Knowledge Building.
212
Because of the complexity of results of this study a comprehensive overview of all results will be
provided next to ensure that the reader has an understanding of all details and the flow of results
to gain the big picture of the relationships across findings. Results will be further elaborated
upon in the next chapter, the Discussion chapter. Study conclusions, will also be framed by the
three clusters of research subquestions: (a) traditional outcome measures; (b) performance “over
and above” traditional measures (beyond learning as traditionally conceived and measured); and
(c) questions regarding the sociocognitive dynamics that enable work over-and-above traditional
learning.
7.4 Overview of Results
Design of the online environment and evaluation tools are validated in the 2004/2005 pilot study.
Minor changes were made, particularly to pain pre-/posttest structure for future iterations.
Traditional assessment measures, including pre-/posttests, attitude and opinion surveys, and
online activity statistics are used to assess outcomes from 2005 to 2008. Assessment is expanded
to include social network and Knowledge Building analyses of the 2008/2009 discourse space.
Results of all findings are summarized below.
7.4.1 Pilot Study 2004/2005
Key results from the pilot study were:
· Pre-/posttest was validated. Specific questions were refined to improve reliability.
· Online activity over five modules of percent of notes read was high ranging between
74-85%; number of total notes contributed was 306 and percent of notes lined were
strong, between 19% and 72%.
· Attitudes and opinions of were very positive. Overall, 80% of respondents rated the
Knowledge Building component of this program as above average or excellent.
7.4.2 Cluster 1: Traditional Measures
7.4.2.1 Matched results of pain knowledge pre-/posttests, 2005–2009.
· 2005/2006 total mean score across all objectives (1 to 4) demonstrated a significant
improvement in pain knowledge of 8%, from 73% on pretest to 81% on posttest, on
paired t-test = 3.48, p < .006. Cohen’s d demonstrated a large effect size of .87.
213
· 2006/2007 total mean score across all objectives (1 to 4) demonstrated strong
improvement in pain knowledge of 10%, from 68% on pretest to 78% on posttest;
however this was not significant (probably due to small sample size) on paired t-test =
2.15, p < .075; Cohen’s d demonstrated a large effect size of .93.
· 2007/2008 total mean score across all objectives (1 to 4) demonstrated a significant
improvement in pain knowledge of 7%, from 68% on pretest to 75% on posttest, on
paired t-test = 2.22, p < .04. Cohen’s d demonstrated a medium effect size of .64.
· 2008/2009 total mean score across all objectives (1 to 4) demonstrated a significant
improvement in pain knowledge of 14%, from 67% on pretest to 81% on posttest, and
on paired t-test = 4.30, p < .001. Cohen’s d indicated a very large effect size of 1.15.
· 2005–2009 cumulative total mean score across all objectives (1 to 4) demonstrated a
significant improvement in pain knowledge of 9%, from 69% on pretest to 78% on
posttest, and on paired t-test = 5.34, p < .001. Cohen’s d indicated an overall large
effect size of .82.
7.4.2.2 Attitude and opinion results 2005–2009
Online collaborative Knowledge Building was rated above average/excellent by 86.7%
respondents. Ninety-two percent agreed/strongly agreed that they would like to see more
distance education program for CME.
7.4.3 Cluster 2: Performance Over and Above Traditional Measures
7.4.3.1 ATK measures 2005–2009.
· In 2005/2006 participants read a mean of 86.3% of notes across all modules and
wrote an average of 18.67 notes per module;
· In 2006/2007 participants read a mean of 92.3% of notes across all modules and
wrote an average of 33.55 notes per module;
· In 2007/2008 participants in Group 1 read a mean of 92.6% of notes and wrote an
average of 30.93 notes per module; participants in Group 2 read a mean of 92.4% of
notes across all modules and wrote an average of 18.25 notes per module; and
· In 2008/2009 participants in Group 1 read a mean of 79.33% of notes and wrote an
average of 36.82 notes per module; while participants in Group 2 read a mean of
214
85.7% of notes across all modules and wrote an average of 25.1 notes per module.*
This year was selected for further social network analysis.
· Facilitator note contributions and build-ons more than twice that of most students;
· SNA of who built-on whose notes is fan shaped with facilitator as pivotal point and
arch shaped indicative of students building-on each other’s ideas; network edges =
25;
· Number of notes read and SNA of who read whose notes was well distributed;
network edges = 110
7.4.3.2 Social network results (KF analytic tools) 2008/2009: Groups 1 and 2.
7.4.3.2.1. Mr. Singh’s Pain, Part 1: Group 1.
· Facilitator note contributions and build-ons approximately twice that of most
students, except one;
· SNA of who built-on whose notes is becoming more distributed around the fan shape
with facilitator as pivotal point; network edges = 27;
· Number of notes read and SNA of who read whose notes was well distributed;
network edges = 100
7.4.3.2.2 Mr. Singh’s Pain, Part 2: Group 1.
· Facilitator note contributions and build-ons approximately twice that of most
students, except one;
· SNA of who built-on whose notes is becoming more distributed around the fan shape
with facilitator as pivotal point; network edges = 27;
· Number of notes read and SNA of who read whose notes was well distributed;
network edges = 100
7.4.3.2.3 Mary’s Misery: Group 1.
· Facilitator note contributions and build-ons approximately twice that of students;
· SNA of who built-on whose notes is becoming more distributed around the fan shape
with facilitator as pivotal point; network edges = 21;
· Number of notes read and SNA of who read whose notes was well distributed;
network edges = 91.
215
7.4.3.2.4 Judy’s Last Days, Part 1: Group 1.
· Facilitator note contributions and build-ons more than three times that of most
students; reading measures are high for all participants;
· SNA of who built-on whose notes fan shaped with facilitator as pivotal point;
network edges = 14;
· Number of notes read and SNA of who read whose notes was well distributed;
network edges = 76
7.4.3.2.5 Judy’s Last Days, Part 2: Group 1.
· Facilitator note contributions and build-ons more than twice that of most students;
reading measures are high for all participants;
· SNA of who built-on whose notes fan shaped with facilitator as pivotal point;
network edges = 20;
· Number of notes read and SNA of who read whose notes was well distributed;
network edges = 66.
7.4.3.2.6 Mr. Singh’s Pain, Part 1: Group 2.
· Facilitator note contributions and build-ons more than twice that of most students;
only 8 people are building-on but 12 are reading.
· SNA of who built-on whose notes is somewhat distribute; network edges = 18;
· Number of notes read and SNA of who read whose notes was well distributed;
network edges = 67
7.4.3.2.7 Mr. Singh’s Pain, Part 2: Group 2.
· Facilitator note contributions and build-ons more than three times that of most
students.
· SNA of who built-on whose notes is distributed in an arch pattern; network edges =
21;
· Number of notes read and SNA of who read whose notes was well distributed;
network edges = 55
7.4.3.2.8 Mary’s Misery: Group 2.
· Facilitator note contributions and build-ons was approximately twice that of most
students; reading of notes was high for almost all participants.
216
· SNA of who built-on whose notes is somewhat distribute; network edges = 12;
· Number of notes read and SNA of who read whose notes was well distributed;
network edges = 54
7.4.3.2.9 Judy’s Last Days, Part 1: Group 2.
· Facilitator note contributions and build-ons more than twice that of most students.
· SNA of who built-on whose notes is distribute amongst few participants; network
edges = 6;
· Number of notes read and SNA of who read whose notes was well distributed;
network edges = 17
7.4.3.2.10 Judy’s Last Days, Part 2: Group 2.
· Facilitator note contributions and build-ons approximately twice that of most
students; reading level was high.
· SNA of who built-on whose notes is distribute amongst few participants; network
edges = 7;
· Number of notes read and SNA of who read whose notes was well distributed among
the few remaining participants; network edges = 31
7.4.3.2.11 Results of 2-way ANOVA on 2008/2009 Groups 1 and 2. Overall, the difference
between the 2008/2009 matched pre- and posttest means combined over group was significant, F
(1,16) = 17.97, p < .001. Overall difference between groups was not significant. Both groups
were strong; both groups demonstrated a similar effect size, of 1.11 and 1.12 respectively.
However, Group 1 demonstrated greater increase and higher level knowledge gains than Group
2. Group 1 demonstrated a 16% improvement in total pain knowledge, from 68% on pretest to
84% on posttest. In comparison Group 2 scores increased 12% from 66% on pretest to 78% on
posttest. Although both groups were strong, Group 1 was deemed to be stronger.
7.4.3.2.12 Network links and density measures and tests of significant difference and
effect size. Comparison between groups (with facilitator) was conducted. Groups 1 mean
network density of build-ons (0.278) was higher than Group 2 (0.182). Group 1 network density
measures of notes read (0.798) was higher than Group 2 (0.657). When facilitator was removed,
SNA results indicated groups remained strong in network build-ons and reading. On t-test and
analysis of effect size, results demonstrated that there was no significant difference on density or
217
effect size of notes build-on and read with and without facilitator. Therefore, student
participation was strong and self-sustaining.
7.4.3.2.13 Social network measures of centrality: Eigenvector, in-degree, and out-degree;
and results of t-test and effect size. Mean Eigenvector centrality measures of build-ons and notes
read are low in both Groups 1 and 2. These low measures are indicative of well-distributed
groups. Eigenvector analysis of groups with and without facilitators demonstrates little change
and is therefore indicative of strong student networks. In-degree/out-degree measures of Group 1
and Group 2 build-ons dropped markedly in some modules when the facilitator was removed
indicating good distribution across the student population, yet high degree of facilitator
participation.
Significant difference and large effect size were found for Group 1 and 2 when tested with and
without Facilitator, on social network centrality dimensions of in-degree and out-degree of notes
built-on. Therefore we can conclude the Facilitators in Groups 1 and 2 were highly participative
in terms of receiving and linking or building-on others notes and had a large effect.
7.4.3.2.14 Significant difference between groups in social network density measures.
Comparison results between Groups 1 and 2, with a facilitator, across all five modules indicates
that density of notes built-on is significantly different. Difference in density for notes built-on
was significantly stronger in Group 1 (p<.01) and effect size was particularly large measuring,
1.31. Density of build-on notes appears to be the major factor emerging from SNA in this case
study that is related to higher levels of knowledge.
Further SNA analyses only focused on build-ons notes (not notes read) including facilitator since
differences were found across groups; notes read measure was very high for both groups and
showed little difference.
7.4.3.2.15 Clique members and clique cohesion index. Group 1 had almost twice as many
total cliques (27) as Group 2 (14). Most striking was the fact that the Group 1 facilitator
participated in all 27 cliques, which is more than three times as many as the Group 2 facilitator,
who participated in 8 of 14 cliques. Despite the very high level of the Group 1 facilitator’s
engagement, the cohesion index across modules was lower than Group 2, indicative of a more
highly distributed and inclusive social networks. Interclique connectivity was maintained in both
groups by the numerous participants that were active across many cliques. “This bridging
218
phenomenon provides for a wealth of information flowing to all members” (Aviv et al., 2003, p.
13).
7.4.3.2.16 Relationship of SN structural analysis of three pain modules to pre-/posttest
knowledge improvement scores. When Groups 1 and 2 were compared, relationship patterns
indicated that higher performance on Group 1
pre-/post knowledge tests were related to higher levels of build-on notes in KF, higher number of
network edges, significant difference (p < .001) in density of build-on notes, lower cohesion
index and centrality measures, as well as a greater number of cliques and facilitator engagement,
in an opportunistic and evenly distributed network.
When Groups 1 and 2 were combined for Spearman nonparametric correlations of significance
of the posttest score with the social network centrality variables, it was found that of the nine
correlations, all are positive and all are above .2, six above .3, and four are significant. In-degree,
out-degree, and Eigenvector centrality measures in various pain modules were significantly
correlated with posttest scores.
7.4.4 Cluster 3: Social Network Analyses and Sociocognitive Dynamics That Support
Knowledge Building Over and Above Learning
7.4.4.1 Results of social network position/power analysis 2008/2009. Network position
(centrality of facilitator/students and distribution of students) and relationship to network power
(prestige and dominance/democracy) although related to SN structural analysis is considered
separately and so that results can be more clearly linked to social network semantic analysis.
The greatest “gainers” on pain knowledge pre-/posttest in the Group 1, shifted positions over
modules and were positioned in the midfield and the periphery more often that in the core; the
greatest “gainer” in Group 2 never occupied a position at the core—only in the mid and
peripheral fields. It appears that there is much to be gained from work at mid field and periphery
positions in a CME context.
It was posited that students with the highest scores on pain pretest would often assume the core
position, sharing this position with the facilitator, which in fact was the case for some but not all.
7.4.4.2 Results of social network content analysis 2008/2009. Results summaries of the
following series of within note analyses are presented below:
219
· Facilitator/students patterns of discourse;
· Emergent themes of discourse (beyond learning objectives);
· Social network complexity of clinical discourse
· Results of analysis of Knowledge Building indicators and exemplars.
7.4.4.2.1 Facilitator/students patterns of discourse patterns. Strongest Group 1 Facilitator
discourse pattern is as partner and partner/expert; strongest Group 2 Facilitator discourse pattern
is as monitor and partner/expert. In regards to statement/questions patterns, students and
facilitators at the core of the social network most frequently made statements or made statements
followed by a question(s). Overall, student/facilitator discourse patterns are striking in that
collective knowledge work proceeds not by interchange of statements alone; but most often
through statements followed by a question(s). Group 1 Facilitator posed more than 50% more
questions in both modules examined, than Group 2 Facilitator.
7.4.4.2.2 Results of analysis of complexity of discourse. Social network analysis of
complexity of clinical discourse was similar across all modules in that almost all student and
facilitator discourse was rated as complex or elaborated. Since all participants in this study are
family physicians, it is not surprising that all clinical discourse reflects a high level of
complexity, within notes, across all four modules scored.
7.4.4.2.3 Emergent themes/threads in knowledge work (beyond predefined learning
objectives). Emergent themes were identified in Group 1 and Group 2 discourse. Emergent
knowledge work can be characterized as adductive and abductive Knowledge Building, working
to improve knowledge so that it is understood more deeply and broadly. Group 1 and Group 2
identified different themes for knowledge work according to their needs and interests. Analogous
cases, “my experience,” “in my practice,” emotions, and pragmatics of practice (e.g., OHIP
billings). Personal knowledge lacks were identified, “need to know” questions were posted,
misconceptions corrected, and grey areas discussed that have no one answer (e.g., the art and the
science of palliative care practice).
7.4.4.2.4 Results of Knowledge Building indicators within social network discourse. All
Knowledge Building principles were evidenced in each module scored and most were found to
be “strong and consistent.” Exemplars of principles of community responsibility, democracy,
220
improvable ideas, epistemic agency demonstrated primarily through emergent themes/ideas, are
indicative of abductive/adductive Knowledge Building.
This comprehensive summary of results provides a big picture of the relationships across
findings. Results will be further elaborated in the next chapter, the Discussion chapter and study
conclusions, will also be framed by the three clusters of research subquestions: (a) traditional
outcome measures; (b) performance “over and above” traditional measures (beyond learning as
traditionally conceived and measured); and (c) questions regarding the sociocognitive dynamics
that enable work over and above traditional learning.
221
CHAPTER 8
DISCUSSION
8.1 Introduction
The chapter begins with a summary of the research, followed by a discussion of study results
based on relevant literature previously reviewed. Discussion of results will be organized
according to the three study clusters, related research questions, and findings.
8.2 Summary of Research
The current study was conducted in a continuing medical education End-of-Life Care Distance
Education course, for family physicians, from 2004 to 2009. A mixed methods case study
methodology was used to determine if socially mediated Knowledge Building improved
physicians’ knowledge, and if so, what social network structural relationships and sociocognitive
dynamics support knowledge improvement, democratization of knowledge, and meta-design of
Knowledge Building in continuing medical education.
Results from this multiyear study supports the use of a Knowledge Building approach in
continuing medical education, aimed at improving individual outcomes through community
opportunities for collective knowledge work, over and above traditional learning (Bereiter &
Scardamalia, 2003; Scardamalia & Bereiter, 2003a, 2005, 2006). Bereiter & Scardamalia (2007)
stated: “We do not suggest that ‘design mode’ should replace ‘belief mode.’ . . . Both are
important. . . . Knowledge creation depends on moving back and forth between belief and design
questions in ways that maintain progress in idea improvement” (p. 26).
The goal of this study was to show improvements according to traditional as well as
nontraditional measures—to show that there can be improvements, not tradeoffs, on both fronts.
The End-of-Life Care Distance Education program demonstrated consistently strong individual
learning and collective outcomes in the pilot and across 4-years. In the final year of the study,
2008/2009 social network structural measures were examined to describe the positive findings
and shed light the relationships across variables that support community Knowledge Building.
These relationships included high network density of build-on notes, well-distributed measures
centrality, and strong group coherence. Further in-depth examination of sociocognitive dynamics
showed shared facilitator/student core position and shared power of idea centrality. Within note
222
analysis clarified facilitator/student discourse structure of partner/expert and sociocognitive
dynamics of idea complexity and evidence of emergent themes for knowledge work, beyond the
learning objectives. Further content analysis demonstrated the presence of numerous Knowledge
Building indicators, such as community responsibility, idea improvement, epistemic agency, and
democratization of knowledge.
A summary of results is provided in Table 53. Discussion of results, based on the previous
literature review, will be similarly organized around the three study clusters: (a) traditional
outcome measures; (b) performance over and above traditional measures (beyond learning as
traditionally conceived and measured); and (c) sociocognitive dynamics that enable work over
and above traditional learning. Discussion of results of subquestions in this chapter, are used to
inform my answer to the research question in the Conclusions chapter.
Table 53
Clusters Summary of Research Questions and Results of Analyses
RESEARCH QUESTION
Does Knowledge Building improve physicians’ knowledge and understanding of palliative care in a Web-based
continuing medical education course; and if so, what social network structural relationships and sociocognitive
dynamics support knowledge improvement, and contribute to democratization and metadesign of Knowledge
Building in continuing medical education?
SUBQUESTIONS
CLUSTER 1 TRADITIONAL OUTCOME MEASURES
Subquestion 1 Did students’ pain knowledge improve from pre to posttest and if so, is this improvement
significant and what is the effect size?
Data Analysis Results 2005-2009
2-tailed t-test on matched pre-/posttests Significant knowledge improvement (9% on
paired t-test = 5.34, p < 0.001)
Cohen’s d effect size Large effect (0.82)
Subquestion 2 Traditionally continuing medical education is a self-directed or didactic experience; what are
participants’ attitudes and opinions towards collaborative online knowledge building in the
End-of-Life Care Program?
Data Analysis Results 2005-2009
Likert scale and Yes/No questions in
online attitudes and opinions survey
Overall very positive, e.g. 86.7% rated their
online collaborative KB experience as above
average/excellent.
(table continues)
223
Table 53 (continued)
CLUSTER 2 PERFORMANCE MEASURES OVER AND ABOVE TRADITIONAL LEARNING
Subquestion 3 What are participants’ online activity and interactivity measures, by module, by group, by
year, in the 2005-2009 study (in each of the five modules, across two groups, and over four-
years)?
Data Analysis Results
5 online modules, by group, each year;
ATK tools in Knowledge Forum (KF)
2005-2009 overall high levels of read, write,
and build-on activity across all years.
Activity graphs and SNA of online
modules, over two groups using ATK tools
in KF
2008/2009 results demonstrate differences in
network density of build-ons between groups
and high facilitator participation
Subquestion 4 Is there a significant difference in 2008/2009 Groups 1 and 2 pain knowledge scores from
pretest to posttest, and is there a significant difference between groups?
Data Analysis Results 2008/2009
2-way ANOVA Gr. 1 and 2 pre-/posttests Difference over group was significant
F(1,16)=17.97, p<.001; difference between
groups was not; Gr.1 knowledge gains > Gr. 2
Subquestion 5 Pain knowledge improved significantly in both groups; however, knowledge increase was
greater in Group 1 than in Group 2. What are the social network structural differences between
groups that supported increased knowledge improvement and how are these differences
related to centralization and/or democratization of participation and ideas?
Structural SNA Data Analysis (Group 1 and Group 2) Results 2008/2009
Network edges of build-ons and notes read
with/without facilitator = 20 analyses
Network links greater in Gr.1 than Gr. 2
Network density of build-ons and notes read
with/without facilitator = 20 analyses
Network density greater in Gr. 1 (0.278) than
Gr. 2 (0.182); difference in density for notes
built-on significantly stronger in Group 1 and
effect size very large (1.31); no significant
difference with/ without facilitator.
Eigenvector centrality of build-ons and
notes read with/without facilitator = 20
analyses
Centrality measures indicative of well
distributed groups in both Gr. 1 and Gr. 2; ); no
significant difference with/without facilitator.
Indegree/Outdegree centrality of build-ons
and notes read with/without facilitator = 20
analyses
Significant difference and large effect size
found for Gr. 1 and 2 with/without facilitator;
therefore, indicative of a high degree of
facilitator participation.
Cohesion index and cliques of build-ons
notes (including facilitator)
(measures: cohesion of ideas, no. of cliques,
no. of members, etc.)
Gr.1=27 cliques and Gr. 2=14; Gr. 1 facilitator
participated in all cliques; Gr. 2 facilitator only
participated in 8/14. Strong within group
cohesion and bridging between idea clusters
Correlation of social network centrality
variables (of combined groups) and pain
posttest knowledge score (Spearman’s rho)
Centrality correlations with posttest scores
were all positive.
224
Table 53 (continued)
CLUSTER 3 SOCIOCOGNITIVE DYNAMICS THAT ENABLE WORK OVER AND ABOVE
TRADITIONAL LEARNING
Subquestion
6
What are the social network relationships between structural position, power (defined as
centrality of ideas), and knowledge improvement; and how are these relationships reflected in
facilitator/student sociocognitive dynamics of Knowledge Building (beyond learning objectives),
through emergent themes, in complexity of discourse, and by indicators of Knowledge Building?
Data Analyses (Group 1 and Group 2) Results 2008/09
a. NETWORK POSITION AND POWER
(10 measures and mapped visualizations)
SNA of position/power/idea centrality and
relationship to knowledge improvement
Shared core position of power (often/but
not always with students with highest
pretest scores); students and facilitators
exhibit idea centrality
Identification of whose ideas are at the core, mid, or
periphery (facilitator and/or students; Aviv et al.,
2003)
Work in mid and peripheral fields by
greatest gainers (students with greatest
pre-/posttest knowledge gains); associated
with notions of identity.
b. FACILITATOR/STUDENT SOCIOCOGNITIVE DYNAMICS
(40% of the dataset was analyzed)
Facilitator stance Facilitator sociocognitive dynamics were
most often categorized as partner/expert
(Modified protocol of monitor, monitor/mentor,
mentor, participant, participant/expert, or expert
used based on Tabak & Baumgartner, 2004)
Who asks the questions that drive knowledge
work?
Questions, as opposed to statements only,
dominated the discourse. Questions were
asked by students and facilitators.
c. ANALYSIS OF EMERGENT THEMES (OVER AND ABOVE
LEARNING OBJECTIVES)
Categorization of themes and threads based on
predefined learning objectives and emergent ideas
(Zhang et al., 2009)
Emergent themes were identified in the
discourse over and above learning
objectives; numerous emergent themes
were generated by Gr.1 (more than Gr. 2);
Themes broadened and deepened ideas, in
abductive and adduction knowledge work.
d. ANALYSIS OF COMPLEXITY OF DISCOURSE
Use of semantic analysis of clinical discourse scale
(Bordage, 1994; Bordage & Lemieux, 1991)
Discourse was categorized as
elaborated/compiled (as opposed to
reduced/dispersed)
e. ANALYSIS OF KNOWLEDGE BUILDING INDICATORS
Evidence of Knowledge Building principles scored
within notes (Use of scale by Sibbald, 2009 and
definitions by Scardamalia, 2002)
Evidence within and across notes of
Knowledge Building principles, such as
community responsibility, democratization
of knowledge, improvable ideas, epistemic
agency, intentionality to rise-above and
work with emergent ideas.
225
This case study was conducted in “Pasteur’s quadrant” defined as “use-inspired basic research,”
which lies at the intersection between pure basic research, aimed at improving understanding,
and purely applied research and development, aimed at improving technology (Stokes, 1997, p.
88). Outcomes of this study will be discussed in the next sections, as well as implications for
next-generation Knowledge Building and Knowledge Forum assessment tools.
8.3 Significance of Cluster 1 Research Results: Traditional Learning Outcomes
and the Relationship to Continuing Medical Education,
Traditionally Conceived
Continuing medical education, as traditionally conceived, has been harshly criticized as being
ineffective, as we have seen demonstrated by numerous papers in the literature review, and
particularly by Davis et al. (1999). In response, continuing medical education is currently being
reconceptualized and redefined. New strategic plans are emerging to address issues of
effectiveness and some, like the University of Toronto (2011) are advocating greater focus on
community and creative knowledge work, while others have redefined the term, to help reframe
objectives; for example, like Price, Havens, and Bell (2011) now refer to continuing medical
education as continuing professional development and improvement—a term that places the
focus beyond learning. Despite this, government regulatory bodies are slow to follow and
continue to defined continuing medical education in terms of individual knowledge improvement
and reward individual accomplish with learning activity credits, as opposed to performance
improvement credits (Dunikowski, 2011). Hence, the traditional measures of knowledge
improvement and program satisfaction, as used in the initial phases of this study, remain
important. Change is slow. As indicated by the results in this research cluster, positive
knowledge gains were demonstrated along with positive feedback on various socially mediated
aspects of Knowledge Building in the End-of-Life Care Distance Education Program across all
years.
Despite these positive outcomes measures, it is evident that to only measure individual outcomes
of knowledge improvement, in a course that focuses on collective improvement and community
knowledge work, only measures some of the outcomes - the traditional learning outcomes, and
not the community outcomes.
226
8.4 Significance of Cluster 2 Research Results: Social Network
Performance Measures Over and Above Learning
Knowledge Building in this continuing medical education course has been described as
successful in terms of knowledge gains demonstrated on pre-/post-test, positive attitudes
and opinions. Beyond those traditional measures, are measures of online activity and
interactivity of collective knowledge work, and most importantly in this study, social
network analysis measures that describe relationships of community engagement in terms of
structure, power, and sociocognitive dynamics of collective knowledge work. Across all 4-
years, the 2008/2009 cohort demonstrated the largest knowledge gains on pre-/posttest
scores and were therefore chosen for further analysis, to explore what made them
successful, not only in their individual knowledge improvement scores, but as a
collaborative Knowledge Building community.
Results of high online read/write/build-on activity and interactivity measures are common to
Knowledge Building research. However, those descriptions barely scratch the surface of the
intricate relationships revealed through social network structural and content analyses, as
found in this study. In the current study, almost all participants read notes, most contributed,
but only some “built-on.” Build-on notes were seen to be intentionally aimed at Knowledge
Building and became the central focus of this study. Strong relationships were demonstrated
between pain knowledge improvement and social network structural measures across groups
in 2008-09. When Group 1 and Group 2 were compared, relationship patterns indicate that
higher performance on Group 1 pre-/post knowledge tests were related to higher levels of
build-on notes in KF, higher number of network edges, significant difference in density of
build-on notes, lower coherence index and centrality measures, as well as, a greater number
of cliques and facilitator engagement, in an opportunistic and evenly distributed network.
The significance of these social network measures is that describe dimensions of collective
engagement and relationships between structures and how the community works with
knowledge and ideas. As we have seen, a paucity exists in the literature in regards to
sustained Knowledge Building studies of continuing medical education that describe online
communities using social network analysis. Punja’s research study (2007) only brief
examined Knowledge Building in this context and Mylopoulos’ study (2007) focused on
physicians in everyday practice. Neither of these studies used social network analysis as a
227
community measure. Both studies expressed concern about physician attitudes toward
creative knowledge work.
Community measures of improvement or success are not typically employed in continuing
medical education, or in education on the whole. However, with the new CanMEDS
competencies for physicians (Frank, 2005), of collaborator and scholar, there may be reason
to believe that measure of collective knowledge work using tools such social network
analysis may become necessary. Similarly, as seen in the literature, recent
recommendations, like the University of Toronto, Faculty of Medicine, Office of CEPD’s
strategic plan that aim to create more opportunities for collective work and the use of
technology, will likely also need new measure to support collective performance assessment
(University of Toronto, 2011).
The literature reviewed in regards to this cluster provides perspectives on Knowledge
Building theory, Knowledge Forum analytical tools, and social network analyses, with
relevant research studies drawn from the domains of education, the social sciences, and
medicine. This study makes a unique contribution to the research literature across domains
of Knowledge Building and social network analysis and continuing medical education.
Emergence is a central idea and indicator of Knowledge Building, and one that distinguishes it
from learning (Bereiter, 2002a; Bereiter & Scardamalia, 1996). Emergent ideas and design mode
work in the End-of-Life Care Distance Education Program can be characterized as authentic,
working with real ideas and real problems (Bereiter & Scardamalia, 2003; Scardamalia, 2002).
Although the cases created are based on real world cases and the objectives identified relate to
real-world problems they are considered 1-step removed since they are predetermined and
instructor-developed, as opposed participant-developed, emergent ideas. However, as
demonstrated in this study one groups’ knowledge work was almost evenly split between
working with predetermined objectives and community emergent ideas.
Traditional learning environments are often far removed from real-world practice. This
disconnect in continuing medical education is one reason why efforts have shown little effect in
changing physician performance (Davis et al., 1999). It can be argued that with opportunities for
authentic discourse there is less of a leap from the world of education to practice—and less
228
concern for current remedies for far-removed endeavours. The area knowledge translation was
specifically created to address this issue (Graham et al., 2006; Davis, Evans et al., 2003).
In “Weaving the Web,” creator of the World Wide Web, Tim Berners-Lee (1999), reflected:
I have a dream for the Web . . . and it has two parts.
In the first part, the Web becomes a much more powerful means for collaboration
between people. I have always imaged the information spaces as something to which
everyone has immediate and intuitive access, and not just to browse, but to create. . . .
In the second part of the dream, collaborations extend to computer. Machines become
capable of analyzing all the data on the Web—content, links, and transactions between
people and computers. A “Semantic Web,” which should make this possible, has yet to
emerge… (pp. 157–158)
Teplovs and Scardamalia’s (2007) recent work on visualization of assessments is evidence that
we are moving into an era of the Semantic Web; again tools are becoming necessary for analysis
of collective online work. Social network analysis tools have been recently integrated in
Knowledge Forum for use not only by researchers but by participants too. Future studies will
explore the use of these social network tools by participants involved in collective Knowledge
Building to monitor and assess their own processes and performance in relationship to others in
the network.
In the End-of-Life Care Distance Education study social network patterns across time (five
months) and databases (Group 1 and Group 2) demonstrated differences in structure and
distribution, both of knowledge/ideas, and participation. Each of the 10 network modules
demonstrated its, own unique patterns and relationships. Each had its own unique set of
characteristics.
Importantly, parallel and overlapping patterns of social network structural results became
evident. This resonance of findings created relationships across measures and attributes to
describe the network. For example, Network density of ideas proved to be very informative and
produced positive parallel findings of distributed centrality measures and cohesion. Hence these
datasets capture, represent, and help us visualize how social networks are structured and how
ideas are connected and worked with.
An interesting parallel resides within Knowledge Forum technology itself; it provides views of
notes and note connections in each view, or problem space. In Knowledge Forum, a threaded
discourse view can be almost instantaneously changed to a network view, which not only
demonstrates participation connections, who built on whose notes, but also relationships between
229
ideas, and clusters of ideas. Hewitt (2001) in a 1997 presentation published in 2001 as “Beyond
Threaded Discourse,” used Knowledge Forum technology to visually represent ideas as networks
(as opposed to threads), for analysis in his study. Knowledge Forum technology enabled easy
and transparent visual identification of idea clusters. Hewitt identified the importance of this type
of technological support in the early phase of development of Knowledge Forum, when the
technology was still WebCSILE. In his paper he stated:
First, enhancements are needed to facilitate idea linking and connectivity. Second,
collaborative group processes need to become more visible. Specifically, a set of tools are
required that permit individuals to better monitor communal knowledge-building activity
and their own participation in that activity. (Hewitt, 2001, p. 207)
Over 10 years later Hewitt’s vision became a reality—realized through the creation of social
network work and other important tools, for participant and researcher use, in Knowledge
Forum.
Numerous social network analyses were conducted in the current study and tools were required
beyond those available in Knowledge Forum. Although network density was found to be an
extremely informative measure in this study, it is currently the only tool available in Knowledge
Forum. Based on the experience of this study, I recommend expanding the suite of social
network tools to include measures of network centrality, cohesion, and cliques.
8.5 Significance of Cluster 3 Research Results: Sociocognitive Dynamics
That Enable Work Over and Above Learning
Discussion of results in this cluster will begin with an overview of facilitator/participant
structures and then focus on principles and sociocognitive dynamic indicators of Knowledge
Building (Scardamalia, 2002), associated network measures, and content analysis results.
8.5.1 Facilitator/Participant Sociocognitive Dynamics
Student Knowledge Building opportunistic networks characteristically do not have a
prominently engaged facilitator; however, in this continuing medical education course one
facilitator was found to be extremely engaged, involved in all cliques; she assumed the role
of “partner or partner/expert” in the discourse, without dominating or controlling the
discourse, enabling democracy and emergence.
230
Although facilitators were persistently positioned in the central power position in the
network, that position was almost always shared with students. Surprisingly, greatest
knowledge “gainers” on pre-/posttests, were not always found working at the core, but
sometimes at the mid- and periphery of the problem space. Similarly, contrary to
expectations, it was posited that students with the highest scores on pain pre-test would
assume the core position, sharing this position with the facilitator, which in fact was the
case some of the time, but not always. It appears that the best and the brightest do not
always want to be in the position of power and authority and teach others; some assume
positions at the periphery and are relatively silent.
Social network centrality maps and measures of power of ideas and network position proved
to be an interesting and informative way to assess relationships within the community. This
is a promising avenue of further Knowledge Building research and builds-on previous
research by Aviv and colleagues (2003) on cohesion, role, and power; as well as Cornelius
and Herrenkohl paper on how power shapes relationships between students and ideas
(2004); and Tabak and Baumgartner’s (2004) work on teacher/student structures; and
Ligorio’s (2009, 2010) research on identity.
8.5.2 Belief- and Design-Mode Knowledge Work
Family physicians register in End-of-Life Care Distance Education Program because they are
interested in improving their knowledge and learning “facts” about palliative care. Needless to
say, there are many important facts to be learned, e.g. how to titrate pain medications. However,
what we saw in the discourse was not only the mathematics of pain medication titration, but
ideas around treatment. Since each patient reacts to different pain medications differently, good
pain management, like many aspects of medicine, is both a science and an art. Hence,
opportunities, beyond learning, for design mode work are required.
Design mode work also became evident in the real-world clinical cases participants posted for
discussion. Use of analogies (Thagard, 1997; Holyoak & Thagard, 1995; Glick & Holyoak,
1980) and abductive reasoning (Thagard & Shelley, 1997) in this way is important to create
deeper understanding and personal meaning (Bruner, 1990).
Thagard posits that “We can judge that a scientific theory is progressively approximating the
truth if it is increasing its explanatory coherence is two key respects: broadening by explaining
231
more phenomena, and deepening by investigations layers of mechanisms” (Thagard, 2007, p.
28). Broadening and deepening ideas, through abductive and adductive knowledge work,
respectively, has important possibilities for Knowledge Building and provides evidence of the
sociocognitive dynamics of the principle of improvable ideas, described as:
Improvable ideas. Sociocognitive dynamics. All ideas are treatable as improvable.
Participants work continuously to improve the quality, coherence, and utility of ideas. For
such work to prosper, the culture must be one of psychological safety, so that people feel
safe in taking risks—revealing ignorance, voicing half-baked notions, giving and
receiving criticism. (Scardamalia, 2002, p. 78)
Further research is needed on the use of analogies and abductive and adductive reasoning, and
implications for coherence of ideas to improve ideas in collective Knowledge Building.
8.5.3 Knowledge Building Principles and Metadesign Concepts
As we have seen in the End-of-Life Care Distance Education study opportunities for
collaborative participation are fundamental to design mode knowledge work. In an emergent
problem space continuing medical education participants have opportunities to be involved at the
metadesign level of curriculum—normally reserved for instructors, teachers, experts, etc. The
Knowledge Building principles of rise-above and epistemic agency drive this process forward.
Scardamalia (2002) defines these principles below.
Rise above. Sociocognitive dynamics. Creative knowledge building entails working
toward more inclusive principles and higher-level formulations of problems. It means
learning to work with diversity, complexity, and messiness, and out of that achieve new
syntheses. By moving to higher planes of understanding, knowledge builders transcend
trivialities and oversimplifications and move beyond current best practices (Scardamalia,
2002, p. 79).
Epistemic agency. Sociocognitive dynamics. Participants recognize both a personal and a
collective responsibility for success of knowledge building efforts. Individually, they set
forth their ideas and negotiate a fit between personal ideas and ideas of others, using
contrasts to spark and sustain knowledge advancement rather than depending on others to
chart that course for them. Collectively they deal with problems of goals, motivation,
evaluation, and long-range planning that are normally left to teachers or managers
(Scardamalia, 2002, p. 79).
In the context of continuing medical education, participant identification of ideas that require
work and improvement in their specific practice context is particularly important, for authenticity
and relevance. Few continuing medical education curricula enable participant identification of
ideas to emerge, let alone “rise above” the prescribed curriculum. Scardamalia’s concept of rise-
232
above specifically addresses the problematic notion of regarding best practices as an end-state
object—as dogma—that cannot be revised.
Epistemic agency “refers to the amount of individual or collective control people have over the
whole range of components of knowledge building—goals, strategies, resources, evaluation of
results, and so on” (Scardamalia and Bereiter, 2006, p. 106). This description is the opposite of
typical, “top-down” designed, structured, didactic, continuing medical education where students
are given little to no control over these components, and definitive hierarchies of power are
evident. Epistemic agency relates to the concept of metadesign, where one is able to take control,
or collaboratively participate in all high level planning, processes, creation and design.
8.5.3 Collective Responsibility and Democratizing Knowledge
Scardamalia & Bereiter (2006) posit that one of the key limitations to Knowledge Building is
when a community is structured with the teacher as the hub of information. Altering that
structure and flow of information from teacher to students, which is classically hierarchical and
didactic, is essential.
As evidenced in the study herein, shifts in power structure, flow of information, and epistemic
agency can be demonstrated by change in network patterns over time. In the End-of-Life Care
Distance Education study we demonstrated the centralized but shared tendency of the facilitator
with high levels of community engagement and demonstration of community responsibility for
emergent ideas and design-mode knowledge work. Scardamalia’s (2002) description of the
sociocognitive dynamics of democratizing knowledge coupled with the principle of community
knowledge/collective responsibility, key principles in this study are defined below.
Community knowledge, collective responsibility. Sociocognitive dynamics.
Contributions to shared, top-level goals of the organization are prized and rewarded as
much as individual achievements. Team members produce ideas of value to others and
share responsibility for the overall advancement of knowledge in the community
(Scardamalia, 2002, p. 80)
Democratizing knowledge. Sociocognitive dynamics. All participants are legitimate
contributors to the shared goals of the community; all take pride in knowledge advances
achieved by the group. The diversity and divisional differences represented in any
organization do not lead to separations along knowledge have/have-not or innovator/non-
innovator lines. All are empowered to engage in knowledge innovation.
233
Technological dynamics. These is a way into the central knowledge space for all
participants; monitoring tools assess evenness of contributions and other indicators of the
extent to which all members do their part in a join enterprise (Scardamalia, 2002, p. 80).
Technological dynamics related to sociocognitive dynamics of democratizing knowledge are also
presented for this principle. As indicated by this study the use of social network analyses tools in
Knowledge Forum and other software, proved to be very informative and thus should be
included in the next revision of Knowledge Building indicators.
The Knowledge Building principle of democratizing knowledge can be explored in new ways as
a result of the embedded Knowledge Forum visualization tools. Distributions of power across
ideas and participants can be determined and visually demonstrated as autocratic or democratic.
An autocratic environment, that is characterized as teacher-centred and by knowledge telling,
transmission, and the unidirectional flow of ideas can now be seen, like a snap shot of a point in
time. Similarly, all kind of democratic distributions can be visualized across participates. The
Knowledge Forum visualization tools make these dynamics “tacit” (Polyani, 1987).
The link between democracy and education is not new, and in fact John Dewey wrote an entire
book on its various dimensions, published in 1916. Dewey conceives of what he calls the
“democratic criterion” in education as “to imply the ideal of a continuous reconstruction or
reorganizing of experience, of such a nature as to increase its recognized meaning or social
content, and as to increase the capacity of individuals to act as directive guardians of this
reorganization” (Dewey, 1916, p. 188). Dewey’s “democratic criterion” is reflected in
Scardamalia and Bereiter’s principle of democratizing knowledge.
Fischer (2009a) referred to “democratizing design” as fostering cultures of participation in next-
generation computer-supported collaborative learning systems and explores authoritative and
democratic models. He indicated that the disadvantage of the authoritative model is that it limits
the number of voices heard; the controlling mechanisms suppress broad participation from
different constituencies. Fischer referred to “democratized design cultures” and noted that the
disadvantages of this model were its potential for reducing trust in and reliability of content, and
explosion of information. He indicated that cultures supported by the authoritative model
encourage consumption of polished, finished products. In contrast, emergent democratized
design cultures are characterized by a wider, richer set of cultural forms and practices that
require new forms of computer supported collaborative learning.
234
Democratized design cultures, such as those engendered by knowledge building, attempt to
articulate ideas about education and the trajectory of lifelong learning. Bereiter (2002a) argued
for liberal education to support a knowledge society, and by liberal he means essentially
enculturation. Bereiter (2002a) proposed that “school be a productive part of that society, a
workshop for the generation of knowledge” (p. 12) and that the process of Knowledge Building
continues beyond school, into work and throughout life. This is a fitting aim for continuing
medical education too—that continuing medical education “be a productive part of that society, a
workshop for the generation of knowledge” (Bereiter, 2002a, p. 12)
The proximity of politics and education, and democracy and knowledge, and ideas on
enculturation, brings us to the question of “responsibility,” whether it be civic or professional.
Framed by the context of this study, we can ask: What is a physician’s professional responsibility
to improve knowledge? The answer is disappointing; it seems not much (Mylopolous, 2007).
Other results indicated that physicians do feel professionally responsible to improve quality of
care (Brennan, 2002). However, improving “quality of care” is different—it is much more
limited than taking responsibility for improving knowledge. Physicians may feel drawn to
improve quality of care because they are engaged in care but often remain removed from
knowledge improvement—and this may be due to the way in which continuing medical
education and professional development are current conceived. The democratization of
continuing medical education aims to turn over individual and collective responsibility to
improve knowledge, ideas, and practice—by making continual professional development
personally meaningful and relevant—through participatory engagement and metadesign.
Democracy, shifts in power, flow of ideas, and epistemic agency are not elements usually
assessed in educational environments. However, it is noteworthy that these design mode types of
assessment are newly informative and complement traditional belief mode knowledge
assessments and indicators of change. Changes in democratic relationships captured by new
Knowledge Forum assessments and positive improvement in knowledge are highlighted herein,
in the End-of-Life Care Distance Education study. These exciting new ways of seeing,
demonstrating, and evaluating change has important implications for future research and enables
us to ask new kinds of questions. We will need to ask and investigate questions like: Do the tools
correlate with design mode Knowledge Building and ultimately indicate network areas of “Big
Ideas,” hot spots of Knowledge Building and innovation (Scardamalia & Bereiter, 2005, p. 28)?
235
Tools to analyze distribution and symmetry of power would be helpful for researchers and
participants working in Knowledge Forum.
Michel Foucault (1994) told us that power distributions become evident within discourse.
Foucault was interested in macro sociocultural relationships that are useful for our framing vis à
vis Knowledge Building. Configurations of power, knowledge, and truth constitute discourse for
him. “One of the most productive ways of thinking about discourse is not as a group of signs or a
stretch of text, but as ‘practices that systematically form the objects of which they speak’”
(Foucault, 1972, p. 49). In this sense, a discourse is something which produces something else
(Mills, 1997). Foucault does not consider discourse as something that exists in and of itself that
can be analyzed in isolation, acontextually. A discursive structure can be demonstrated through
“systematicity of ideas,” ways of thinking and behaving in a context, and the “effects” of those
ways of thinking and behaving (Mills, 1997). The effects of discourse important to consider are
the elements of truth, power and knowledge. Truth, interests Foucault, from the perspective of
production and mechanics—how social practices regard questions of truth and authority. Power,
according to Foucault is dispersed throughout social relations and produces possible forms of
behaviour as well as restricts.
Knowledge Building in Knowledge Forum enables us to look at power relations in association
with examination of “truth” and idea advancement. Although the End-of-Life Care Distance
Education study did not examine discourse through this type of text analysis, the framing of
discourse as an object, with characteristics of power relations, effects, and as an evolving system
over time is implicit in the analyses work done herein. Foucauldian ideology may provide an
important framework for future research at the grain of semantics.
Roger Saljo (2003) in a keynote address titled: “Representational Tools and the Transformation
of Learning,” makes the point that new technologies do not enhance learning; however, if they
are powerful enough, they transform basic features of how people communicate knowledge, how
knowledge is organized, and how it is represented. Knowledge Building representations, of ideas
in discussion notes, interaction patterns in databases, and social network assessments for research
and concurrent feedback, have the potential to transform current conceptualizations of continuing
medical education.
236
8.6 Summary
This study advocates based on the results presented herein, pushing the current agenda of reform
for continuing medical education even further forward—to go “beyond best practices”
(Scardamalia, 2002) - to reframe continuing medical education within a Knowledge Building
approach, for intentional cognitive collaborative knowledge work—for meaningful and relevant
lifelong learning and sustained improvement of ideas, expertise, and patient care.
This study provides strong evidence for change and exemplars of social network
collaborative participation, emergent ideas, and metadesign of curriculum. These aspects
provide a community model for collective Knowledge Building to address current
deficiencies in continuing medical education. This is the first study to address
sociocognitive dynamics of a Knowledge Building continuing medical education online
community. The conclusions of this study are presented next.
237
CHAPTER 9
CONCLUSIONS
9.1 Introduction
This chapter includes a series of conclusions summarizing research findings, as well as strength
and limitations of research, and implications and recommendations for future research.
9.2 Research Questions and Answers
In this study I asked:
Does Knowledge Building improve physicians’ knowledge and understanding of
palliative care in a Web-based continuing medical education course, over a five-year
period; and if so, what social network structural relationships and sociocognitive
dynamics support knowledge improvement, and contribute to democratization and
metadesign of Knowledge Building in continuing medical education?
The course on which this research is based is ongoing; the research covers a five-year period,
2004 to 2009, and presents results from a series of investigations based on notes entered into the
course environment, Knowledge Forum, by physicians. Results continue to inform the design of
this course.
Results of this study demonstrate that physicians’ knowledge and understanding of palliative
care improved while working collectively online. Perhaps that could be achieved in any
effectively designed course. In the current context these results were important because they
provided the backdrop for broader goals; namely, to demonstrate advances beyond those that can
be achieved in well-designed courses. Thus the finding that physicians’ knowledge and
understanding of palliative care improved while working collectively online provides the basic
finding. Detailed examination of the social network structural relationships and sociocognitive
dynamics that enabled individual achievement also enabled collective knowledge improvement,
as elaborated below.
This study demonstrated how physicians can create a culture of participation and work
collaboratively to improve knowledge and understanding, and in doing so address not only
238
predefined learning objectives, but work over and above through emergent ideas, to contribute to
the design of the curriculum. This is what I have called democratization of knowledge and
metadesign in this case study. These findings reflect Knowledge Building principles of
community responsibility, epistemic agency, idea improvement, and others. Through these meta
processes physicians’ emergent ideas became a strong focus of discourse; their agenda was
addressed, alongside that of the predetermined learning objectives from the content experts. The
sociocognitive dynamics of Knowledge Building, over and above learning, provide a unique
example in this case study of the potentiality of Knowledge Building democratization and
metadesign of continuing medical education.
To summarize, in this study, traditional pre-/posttest learning measures across 4-years showed
significant gains (9% on paired t-test = 5.34, p < 0.001) and large effect size (0.82). Social
network analysis of ten 2008/2009 modules showed significant difference in density of build-on
notes across groups. Additional results demonstrated a relationship between high knowledge
gains and social network measures of centrality/distribution and cohesion. Correlation of posttest
scores with centrality variables were all positive. Position/power analyses highlighted core-
periphery sociocognitive dynamics between the facilitator and students. Facilitators most often
evoked partner/expert relationships. Questions rather than statements dominated the discourse;
discourse complexity was elaborated/compiled as opposed to reduced/dispersed. Themes beyond
pre-defined learning objectives emerged and Knowledge Building principles of community
responsibility, idea improvability, and democratization of knowledge were evident.
Overall, results of the current study showed positive effects of collective Knowledge Building, as
demonstrated by the social network structural relationships and sociocognitive dynamics that
support a culture of participation, democratization, and metadesign of continuing medical
education, over and above individual accomplishments. The potential of collective Knowledge
Building and design-mode work for continuing medical education is evident, with individual
learning representing an important by-product. As demonstrated in this study, there were no
discernible decrements in individual performance, suggesting significant advantages rather than
tradeoffs from engagement in collective knowledge building.
239
9.3 Strengths of This Research Study
Strengths of the research rest with theory-based design and research to examine a novel approach
to collective Knowledge Building as defined by Scardamalia and Bereiter (2003a) and to explore
sociocognitive dynamics of underlying principles (Scardamalia, 2002). Physician participation in
the End-of-Life Care Distance Education Program over a five-year period, from 2004 to 2009,
provided a robust dataset for the research. To realize these strengths it was necessary to design
an educational program with sufficient power to enable the advances represented in the research.
Thus the course itself provides an important artifact—one that is being continually improved and
implemented in the field of continuing medical education.
The case study research methods implemented allowed for naturalistic examination of the
ongoing discourse of participants, as these discourses were captured in the Knowledge Forum
online environment. All participants agreed to participate in the research component, with
assurances of anonymity (all real names were replaced with pseudonyms—even those in the
exemplar Knowledge Forum notes). This study provided us with an opportunity for authentic,
naturalistic, and holistic research to examine complexities and nuances in situ, as physicians
engaged in learning and knowledge building.
An additional strength was the use of social network analysis to assess and demonstrate
collective outcomes and relationships. Various methods were used to assess different dimensions
and a series of parallel findings emerged, with one supporting the other, enabling interdigitation
of methods and outcomes. For example, this study clarified social network structural dimensions
of network density, centrality, and coherence. Strength of participation, distribution, and
democratization between facilitator and students was further supported by findings of social
network position and power analysis.
Content analyses of notes were used to enrich other analyses and to advance our understanding
of knowledge cocreation and metadesign, facilitator/student sociocognitive dynamics, and
facilitator/participant structures underlying high level facilitator engagement. These analyses
were aimed at a meta level, in the sense that they were designed to enrich understanding of
sociocognitive dynamics, rather than strictly the level of the semantics within the discourse
(Hodges et al., 2008). The strength of results related to sociocognitive dynamics rests ultimately
240
with descriptions of social network relationships that scaffold collective knowledge building, and
the individual measures of learning, as by-product of community achievements.
9.4 Limitations
This study was limited by the number of participants working online each year. Annual and four-
year cumulative 2005–2009 analyses were employed on pre-/posttest and survey results to enable
tests of significance. The sample population was not randomized and/or selected according to
any criteria; all participants, who signed up for the course, were included in this study, if they
provided their consent.
The study focused on the strongest dataset—the groups with strongest outcomes. Thus the
research did not examine weaker groups, ineffective social network structures, or comparisons
between weak and strong networks across years. However, now that patterns associated with
strong outcomes have been identified this next stage in the research program will be greatly
facilitated. In addition, build-on notes were specifically chosen as the only focus of study for
deeper social network analysis of position and power, as build-on notes, by definition, show
linked clusters and they provide a clear basis for examining sociocognitive dynamics. Other
parameters such as note contribution or reading measures will be important to study in the future.
Findings may also be limited by the social network analysis tools used, preset parameters, and
assessments they are programmed to conduct. By the very definition of the course design and
methodology employed, this multiyear case study may not be generalizable to other contexts;
nonetheless, to the extent possible and within the scope of this investigation, mixed methods and
multiple measures were used to demonstrate consistency of findings and reliability of results.
9.5 Future Research
Additional research is needed for more detailed accounts of linkage between content analysis and
social network structural analyses. And as suggested from the account of above, future research
needs to address limitations in the current generation of automated social network and semantic
analysis tools. These tools are developing rapidly and show great potential for advancing
concurrent and embedded assessment in online environments (Lax et al., 2010). As methods are
refined for research they will open up much greater potential for providing more effective
feedback to the formation of knowledge advancing teams.
241
To measure individual learning in a knowledge building community is to measure only one facet
of knowledge work. This research shows that measures of community work and sociocognitive
engagement provide a more systemic approach focused less on patterns of difference among
individuals and more on the differences of patterns of relations among sociocognitive factors
(Salomon, 1994a, 1994b).
Current impoverishment of continuing medical education opportunities for collaborative
engagement and design mode innovation leaves ample room for improvement. Positive outcomes
of this study provide strong incentives for adoption of a knowledge building approach for next-
generation initiatives.
9.6 Final Remarks
Scardamalia and Bereiter (2010) argued that the emerging knowledge age poses an educational
challenge that can best be met by adopting a knowledge building approach and a key feature of
that approach is “democratization of innovation capacity” (p. 11). Unlike research scientists,
most physicians do not see themselves as a community of knowledge builders responsible for
idea advancement and innovation (Mylopoulos, 2007). As this study indicates when physicians
are engaged in a culture of participation and provided with opportunities for sociocognitive
work, they can take responsibility for idea improvement through creative, design mode work,
and work constructively as a Knowledge Building community.
This is the first extensive, multiyear study of Knowledge Building in continuing medical
education that focuses on understanding social network structural relationships and
sociocognitive dynamics, over and above learning. Positive outcomes of this study may inspire
curriculum designers and participants, themselves, to consider educational models that advance
beyond individualistic, teacher-centric learning, toward emergent, design mode dynamics in
environments that offer opportunities to boost collective capacity for change and innovation and
work with emergent ideas.
Adventures with ideas located in one’s head do not provide much opportunity for expansive
knowledge work (Whitehead, 1933, p. 278). When ideas are democratized, located at-the-centre,
objectified in a public space, opportunities for sociocognitive improvement and knowledge
innovation become possible (Bereiter, 2002; Scardamalia, 1999, 2002, 2003a, 2004b;
Scardamalia & Bereiter, 2003a, 2003b). The democratization of continuing medical education
242
may enable individuals to take collective responsibility for going beyond best practice and
evidence-based learning.
Continuing medical education defined as learning is necessary, but not sufficient, as
demonstrated in this study. Working at the edge does not come easy; it requires epistemic agency
and intentionality aimed at improving ideas and sustaining innovation. A new model of design-
mode education may engender a change in attitude and culture, from one that typically values
individual accomplishment to one that values collective achievement—a community that values
knowledge creation as well as knowledge acquisition.
Continuing medical education has been harshly criticized as ineffective; current evaluation and
reconceptualization has led to envisioning the potential of what might be. The importance of
reenvisioning the past to inform the future is captured in Whitehead’s (1933) words. He stated:
A race preserves its vigour so long as it harbours a real contrast between what has been
and what may be; and so long as it is nerved by the vigour to adventure beyond the
safeties of the past. Without adventure civilization is in full decay.
It is for this reason that the definition of culture as the knowledge of the best that has
been said and done, is so dangerous by reason of its omission. It omits the great fact that
in their day the great achievements of the past were the adventures of the past.
. . . A living civilization requires learning; but it lies beyond it. (Whitehead, 1933, pp.
278–279)
Whitehead’s notion of the potential of adventures with ideas to impact civilization is not unlike
Scardamalia & Bereiter’s, on the value of knowledge building to potentially effect change and
contribute to the advancement of knowledge to impact real world outcomes. Scardamalia and
Bereiter (2010) state: “Continual idea improvement is the hallmark of a progressive society, and
the present age calls for this responsibility to be much more widely distributed than before” (p.
12). The results and implications of this study, coupled with identified need for change in
continuing medical education, demonstrate the necessity for ongoing research-based innovation.
243
REFERENCES
Abrahamson, S., Baron, J., Elstein, A. S., Hammond, W. P. , Holzman, G. B. Marlow, B., . . .
Schulkin, J. (1999). Continuing medical education for life: Eight principles. Academic
Medicine, 74(12), 1288–1294. Retrieved from http://journals.lwww.com
/academicmedicine
Accreditation Council for Continuing Medical Education (ACCME©
). (n.d.). A directed self-
assessment for readiness to meet the ACCME’s updated accreditation criteria. Retrieved
from http://www.pamedsoc.org/MainMenuCategories/CMEAccreditation
/Organizations/Tools/Readiness.pdf
Akin, O. (2001). “Simon says”: Design is representation. Unpublished manuscript, School of
Architecture, Carnegie Mellon University, Pittsburgh, PA. Retrieved from http://www.
andrew.cmu.edu/user/oa04/Papers/AradSimon.pdf
American Medical Association (AMA). (2010). The physician’s recognition award and credit
system: Information for accredited providers and physicians. Retrieved from
http://www.ama-assn.org/resources/doc/cme/pra-booklet.pdf
Anderson, L., & Krathwohl, D. A. (2001). Taxonomy for learning, teaching and assessing: A
revision of Bloom’s taxonomy of educational objectives. New York, NY: Longman.
Appelbaum, P. (2002). Here comes lifelong learning. Psychiatric News, 37(19), 3–38.
Association of American Medical Colleges. (2004, July). Educating doctors to provide high
quality medical care. A vision for medical education in the United States (Report of the
ad hoc committee of deans). Washington, DC: Author.
Aviv, R., Erlich, Z., Ravid, G., & Geva, A. (2003). Network analysis of knowledge construction
in asynchronous learning networks. The Journal of Asynchronous Learning Networks,
7(3), 1–23. Retrieved from http://sloanconsortium.org
Barrows, H. S., & Tamblyn, R. M. (1980). Problem-based learning: An approach to medical
education. New York, NY: Springer-Verlag.
Bennett, N. L., Davis, D. A. Easterling, W. E. Jr., Friedmann, Green, P., Koeppen, B. M., . . .
Waxman, H. S. (2000). Continuing medical education: A new vision of the professional
development of physicians. Academic Medicine, 75(12), 1167–1172. Retrieved from
http://journals.lww.com/academicmedicine
Bereiter, C. (2000). Drilling down and building up. The Journal of the Learning Sciences, 9(2),
205–208. doi:10.1207/s15327809jls0902_4
Bereiter, C. (2002a). Artifacts, canons, and the progress of pedagogy: A response to contributors.
In B. Smith (Ed.), Liberal education in a knowledge society (pp. 223–244). Chicago, IL:
Open Court.
Bereiter, C. (2002b). Design research for sustained innovation. Cognitive Studies, Bulletin of the
Japanese Cognitive Science Society, 9(3), 321–327.
244
Bereiter, C. (2002c). Education and mind in the knowledge age. Mahwah, NJ: Lawrence
Erlbaum Associates.
Bereiter, C., Burtis, J., & Scardamalia, M. (1994). Problem-centered knowledge: Education
beyond content and process. Toronto, ON: OISE/UT.
Bereiter, C., & Scardamalia, M. (1989). Intentional learning as a goal of instruction. In L. B.
Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp.
361–392). Hillsdale, NJ: Lawrence Erlbaum Associates.
Bereiter, C., & Scardamalia, M. (1990). An architecture for collaborative knowledge building. In
E. De Corte, M. Linn, H. Mandl, & L. Verschaffel (Eds.), Computer-based learning
environments and problem solving (pp. 41–66). Berlin, Germany: Springer-Verlag.
Bereiter, C., & Scardamalia, M. (1993). Surpassing ourselves: An inquiry into the nature and
implications of expertise. Peru, IL: Open Court.
Bereiter, C., & Scardamalia, M. (1996). Rethinking learning. In D. Olson and N. Torrance
(Eds.), The handbook of education and human development: New models of learning,
teaching, and schooling (pp. 485–513). Malden, MA: Blackwell.
Bereiter, C., & Scardamalia, M. (1999). Beyond Bloom’s taxonomy: Rethinking knowledge for
the knowledge age. In M. Fullan (Ed.), The international handbook of educational
change. Dordrecht, the Netherlands: Kluwer.
Bereiter, C., & Scardamalia, M. (2003). Learning to work creatively with knowledge. In E. De
Corte, L. Verschaffel, N. Entwistle, & J. van Merriënboer (Eds.), Powerful learning
environments: Unravelling basic components and dimensions (pp. 55–68). Advances in
learning and instruction series. Oxford, UK: Elsevier Science.
Bereiter, C., & Scardamalia, M. (2007, January). Toward research-based innovation. In Centre
for Educational Research and Innovation (Ed.), Innovating to learn, learning to innovate
(pp. 67–91). Paris, France: OECD.
Berners-Lee, T., with Fischetti, M. (1999). Weaving the web: The original design and ultimate
destiny of the World Wide Web by its inventor. San Francisco, CA: Harper.
Bielaczyc, K. (2006). Designing social infrastructure: Critical issues in creating learning
environments with technology. Journal of the Learning Sciences, 15(3), 301–329.
doi:10.1207/s15327809jls1503_1
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy
of educational objectives: The classification of educational goals. Handbook 1: Cognitive
domain. New York, NY: Longmans, Green.
Bordage, G. (1994). Elaborated knowledge: A key to successful diagnostic thinking. Academic
Medicine, 69(11), 883–885. Retrieved from http://journals.lww.com/academicmedicine
Bordage, G., Connell, K. J., Chang, R. W., Gecht, M. R., & Sinacore, J. M. (1997). Assessing the
semantic content of clinical case presentations: Studies of reliability and concurrent
245
validity [Supplement]. Academic Medicine, 72(10), S37–S39. Retrieved from
http://journals.lww.com/academicmedicine
Bordage, G., & Lemieux, M. (1991).Semantic structures and diagnostic thinking of experts and
novices [Supplement]. Academic Medicine, 66(9), S70–S72. Retrieved from
http://journals.lww.com/academicmedicine
Bradley, E. H., Cramer, L. D., Bogardus , S. T., Kasl, S. V., Johnson-Hurzeler, R., & Horwitz, S.
M. (2004, April). Physicians’ rating of their knowledge, attitudes and end-of-life-care
practices. Academic Medicine, 77(4), 305–311.
Brennan, T. A. (2002). Physicians’ professional responsibility to improve the quality of care.
Academic Medicine, 77(10), 973–980. Retrieved from http://journals.lww.com
/academicmedicine
Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating
complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2),
141–178. doi:10.1207/s15327809jls0202_2
Brown, A. L., & Palincsar, A. S. (1989). Guided, cooperative learning and individual knowledge
acquisition. In L. B. Resnick (Ed.), Knowing, learning and instruction: Essays in honor
of Robert Glaser (pp. 393–451). Hillsdale, NJ: Lawrence Erlbaum Associates.
Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning.
Educational Researcher, 18, 32–42. doi:10.3102/0013189X018001032
Brown, J. S., & Duguid, P. (2000). The social life of information. Boston, MA: Harvard Business
School Press.
Bruner, J. (1990). Acts of meaning. Cambridge, MA: Harvard University Press.
Burt, R. S. (2004). Structural holes and good ideas. The American Journal of Sociology, 110(2),
349–399. doi:10.1086/421787
Burtis, J. (2001, September 28). Analytic toolkit for Knowledge Forum®
. Retrieved from
http://kf.oise.utoronto.ca/atk/cgis/atkdoc.html
Burtis, J., Chan, C., Hewitt, J., Scardamalia, M., & Bereiter, C. (1993, April). Misconceptions in
a student-guided research project. Poster session presented at the annual meeting of the
American Educational Research Association, Atlanta, GA.
Campbell, C. M., & Parboosingh, I. J. (2011). The evolution of continuing professional
development at the Royal College of Physicians and Surgeons of Canada: Setting
standards for Canadian specialists. In D. K. Wentz (Ed.), Continuing medical education:
Looking back, planning ahead (pp. 227–239). Hanover, NH: Dartmouth College Press.
Carrington, P. J., Scott, J., & Wasserman, S. (Eds.). (2005). Models and methods in social
network analysis. Cambridge, MA: Cambridge University Press.
Cazden, C. B. (1988). Classroom discourse. Portsmouth, NH: Heinemann.
246
Chang, R. W., Bordage, G., & Connell, K. J. (1998). Cognition, confidence, and clinical skills:
The importance of early problem representation during case presentations [Supplement].
Academic Medicine, 73(10), S109–111. Retrieved from
http://journals.lww.com/academicmedicine/
Charon, R. (2004). Narrative and medicine. The New England Journal of Medicine, 350(9), 862–
864. Retrieved from http://www.nejm.org
Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The
Journal of the Learning Sciences, 6(3), 271–315. doi:10.1207/s15327809jls0603_1
Chi, M. T. H. (2005). Commonsense conceptions of emergent processes: Why some
misconceptions are robust. Journal of the Learning Sciences, 14(2), 161–199.
doi:10.1207/s15327809jls1402_1
Christakis, N. A., & Fowler, J. H. (2007). The spread of obesity in a large social network over 32
years. New England Journal of Medicine, 357, 370–379.
Christakis, N. A., & Fowler, J. H. (2008). The collective dynamics of smoking in a large social
network. New England Journal of Medicine, 358, 2249–2258.
Christensen, C. R., Garvin, D. A., & Sweet, A. (Eds.). (1991). Education for judgment: The
artistry of discussion leadership. Boston, MA: Harvard Business School Press.
Chuy, M., Scardamalia, M., Bereiter, C., Prinsen, F., Resendes, M., Messina, R., . . . Chow, A.
(2010). Understanding the nature of scientific progress: A theory-building approach.
Canadian Journal of Learning and Technology, 36(1). Retrieved from http://www.cjlt.ca
Cimino, J. J. (1999). Development of expertise in medical practice. In R. J. Sternberg & J. A.
Horvath (Eds.), Tacit knowledge in professional practice: Researcher and practitioner
perspectives (pp. 102–120). Mahwah, NJ: Lawrence Erlbaum Associates.
College of Family Physicians of Canada. (n.d.). [Website]. Retrieved from http://cfpc.ca/CPD
Collins, J. (2009). Education techniques for lifelong learning: Lifelong learning in the 21st
century and beyond. RadioGraphics, 29, 613–622. doi:10.1148/rg.292085179
Collins, A., Brown, J. S., & Holum, A. (1991, Winter). Cognitive apprenticeship: Making
thinking visible. American Educator/American Federation of Teachers.
Collins, A., Brown, J. S., & Newman, S. (1989). Cognitive apprenticeship: Teaching the craft of
reading, writing, and mathematics. In L. B. Resnick (Ed.), Knowing, learning and
instruction: Essays in honor of Robert Glaser (pp. 453–494). Hillsdale, NJ: Lawrence
Erlbaum Associates.
Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and
methodological issues. The Journal of the Learning Sciences, 13(1), 15–42.
doi:10.1207/s15327809jls1301_2
247
Connell, K. J., Bordage, G., Gecht, M. R., & Chang, R. (1998). Assessing clinicians’ quality of
thinking and semantic competence: A training manual. Chicago, IL: University of
Illinois.
Cook, D. A. The research we still are not doing: An agenda for the study of computer-based
learning. (2005). Academic Medicine, 80(6), 541–548. Retrieved from http://journals
.lww.com/academicmedicine
Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2008).
Internet-based learning in the health professions. Journal of the American Medical
Association, 300(10), 1181–1196.
Cornelius, L. L., & Herrenkohl, L. R. (2004). Power in the classroom: How the classroom
environment shapes students’ relationships with each other and with concepts. Cognition
and Instruction, 22(4), 467–498. Retrieved from http://www.jstor
.org/stable/3233886
Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods
approaches. Thousand Oaks, CA: Sage.
Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five
approaches. Thousand Oaks, CA: Sage.
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods
approaches. Thousand Oaks, CA: Sage.
CSCL2011. (2011, July 4–8). Ninth International Conference on Computer-Supported
Collaborative Learning, Hong Kong [Website]. Retrieved from http://www.isls.org
/cscl2011/index.htm?CFID=36331337&CFTOKEN=28690434
Curran, V., Lockyer, J., Sargeant, J., & Fleet, L. (2006). Evaluation of learning outcomes in web-
based continuing medical education. [Supplement]. Academic Medicine, 81(10), S30–
S34. doi:10.1097/01.ACM.0000236509.32699.f5
Custers, E. J. F. M., Regehr, G., & Norman, G. R. (1996). Mental representations of medical
diagnostic knowledge: A review [Supplement]. Academic Medicine, 71(10), S55–S61.
Retrieved from http://journals.lww.com/academicmedicine/
Dauphinee, W. D. (2011). The evolution of continuing medical education in Canada. In D. K.
Wentz (Ed.), Continuing medical education: Looking back, planning ahead (pp. 205–
217). Hanover, NH: Dartmouth College Press.
Dauphinee, W. D., & Wood-Dauphinee, S. (2004). The need for evidence in medical education:
The development of best evidence medical education as an opportunity to inform, guide,
and sustain medical education research. Academic Medicine, 79(10), 925–930. Retrieved
from http://journals.lww.com/academicmedicine/
Davis, D. (2011). Can CME save lives? The results of a Swedish, evidence-based continuing
education intervention. Annals of Family Medicine, 9, 198–200. doi: 10.1370/afm.1256
248
Davis, D., Barnes, B. E., & Fox, R. (Eds.). (2003). The continuing professional development of
physicians: From research to practice. Chicago, IL: American Medical Association
Press.
Davis, D., & Davis, N. (2010, February 9). Selecting educational interventions for knowledge
translation. Canadian Medical Association Journal, 182(2), E89–E93.
doi: 10.1503/cmaj.081241
Davis, D., Evans, M., Jadad, A., Perrier, L., Rath, D., Ryan, D., . . . Zwarenstein, M. (2003). The
case for knowledge translation: Shortening the journey from evidence to effect. British
Medical Journal, 327, 33–35. doi:10.1136/bmj.327.7405.33
Davis, D., Mazmanian, P. E., Fordis, M., Van Harrison, R., Thorpe, K. E., & Perrier, L. (2006).
Accuracy of physician self-assessment compared with observed measures of competence:
A systematic review. The Journal of the American Medical Association, 296(9), 1094–
1102. doi:10.1001/jama.296.9.1094
Davis, D., Thompson O’Brien, M. A., Freemantle, N., Wolf, F. M., Mazmanian, P., & Taylor-
Vaisey, A. (1999, September 1). Impact of formal continuing medical education: Do
conferences, workshops, rounds, and other traditional continuing education activities
change physician behavior or health care outcomes? Journal of the American Medical
Association, 282(9), 867–874.
Davis, N., Davis, D., & Bloch, R. (2008). Continuing medical education: AMEE Education
Guide #35. Medical Teacher, 30(7), 652–666. doi:10.1080/01421590802108323
Desa, V., Danjoux, C., Matyas, Y., Fitch, M., Husain, A., Horvath, N., . . . Barnes, E. A. (2008).
An interdisciplinary and collaborative initiative in palliative care research. Journal of
Pain Research, 2, 1–3. doi:10.2147/JPR.S4622
Dewey, J. (1997). Experience and education. New York, NY: Touchstone.
Dillman, D. (1978). Mail and telephone surveys: The total design method. New York, NY:
Wiley.
Dillman, D. (2000). Mail and internet surveys: The total design method. New York, NY: Wiley.
Donovan, S. M., Bransford, J. D., & Pellegrino, J. W. (Eds.). (1999). How people learn:
Bridging research and practice. Washington, DC: National Academy Press.
Dorman, T., & Miller, B. M. (2011). Continuing medical education: The link between physician
learning and health care outcomes. Academic Medicine, 86,1339.
doi: 10.1097/ACM0b013e3182308d49
Dreyfus, H. L., & Dreyfus, S. E. (1986). Five steps from expert to novice. In H. L. Dreyfus & S.
E. Dreyfus, with T. Athanasiou (Eds.), Mind over machine: The power of human intuition
and expertise in the era of the computer. (pp. 16–51). New York, NY: The Free Press.
Dunikowski, L. G. (2011). The College of Family Physicians of Canada: Continuing medical
education and continuing professional development in Canada. In D. K. Wentz (Ed.),
249
Continuing medical education: Looking back, planning ahead (pp. 240–247). Hanover,
NH: Dartmouth College Press.
Elstein, A. S. (1999). Heuristics and biases: Selected errors in clinical reasoning. Academic
Medicine, 74(7), 791–794. Retrieved from http://journals.lww.com/academicmedicine
Elstein, A. S., Shulman, L., & Sprafka, S. (1990). Medical problem solving: A ten-year
retrospective. Evaluation and the Health Professions, 13, 5–36.
End-of-Life Care Distance Education. (2004) [Website]. Retrieved from http://icarus.med
.utoronto.ca/eolcare/
Engeström, Y., Miettinen, R., & Punamäki, R.-L. (Eds.). (1999). Perspectives on activity theory.
Cambridge, MA: Cambridge University Press.
Eva, K. W., & Regehr, G. (2005). Self-assessment in the health professions: A reformulation and
research agenda [Supplement]. Academic Medicine, 80(10), S46–S54. Retrieved from
http://journals.lww.com/academicmedicine/
Fineberg, I. C., Wenger, N. S., & Forrow, L. (2004). Interdisciplinary education: Evaluation of a
palliative care training intervention for pre-professionals [Research report]. Academic
Medicine, 79(8), 769–776. Retrieved from http://journals.lww.com/academicmedicine
Fischer, G. (2009a). Democratizing design: New challenges and opportunities for computer-
supported collaborative learning. Proceedings (1) of CSCL’2009: Eighth International
Conference on Computer-Supported Collaborative Learning (pp. 282–286). University of
the Aegean, Rhodes, Greece. Retrieved from http://l3d.cs.colorado.edu
/~gerhard/papers/2009-CSCL-paper.pdf
Fischer, G. (2009b). End-user development and meta-design: Foundations for cultures of
participation. Proceedings of the Second International Symposium on End User
Development (pp. 3–14). Siegen, Germany. Retrieved from http://l3d.cs.colorado
.edu/~gerhard/papers/EUD-siegen-2009.pdf
Fischer, G. (2010, October). Extending boundaries with meta-design and cultures of
participation. Proceedings of the Sixth Nordic Conference on Human-Computer
Interaction, “Extending Boundaries” (pp. 168–177). Reykjavik, Iceland. Retrieved from
http://l3d.cs.colorado.edu/%7Egerhard/papers/2010/nordichi-paper.pdf
Fischer, G., & Giaccardi, E. (2006). Meta-design: A framework for the future of end user
development. In H. Lieberman, F. Paternò, & V. Wulf (Eds.), End user development:
Empowering people to flexibly employ advanced information and communication
technology (pp. 427–457). Dordrecht, the Netherlands: Kluwer Academic. Retrieved
from http://l3d.cs.colorado.edu/~gerhard/papers/EUD-meta-design-online.pdf
Fischer, G. & Konomi, S. (2007). Innovative socio-technical environments in support of
distributed intelligence and lifelong learning. Journal of Computer Assisted Learning,
23(4), 338–350. doi:10.1111/j.1365-2729.2007.00238.x
250
Fordis, M., King, J. E., Ballantyne, C. M., Jones, P. H., Schneider, K. H., Spann, S. J., &
Greisinger, A. J. (2006). Internet-based continuing medical education: Reply. Journal of
the American Medical Association, 295(7), 758–759. doi:10.1001/jama.295.7
.758-b
Foucault, M. (1972). The archaeology of knowledge and the discourse on language. New York,
NY: Pantheon Books
Foucault, M. (2001). Power (J. D. Faubion, Ed.; R. Hurley and others, Trans.). In P. Rabinow
(Series Ed.), The essential works of Foucault, 1954–1984: Vol. 3. New York, NY: New
Press.
Fowler, J. H., & Christakis, N. A. (2008). Dynamic spread of happiness in a large social
network: Longitudinal analysis over 20 years in the Framingham Heart Study. British
Medical Journal, 337. doi: 10.1136/bmj.a2338
Frank, J. R. (Ed). (2005). The CanMEDS 2005 physician competency framework. Better
standards. Better physicians. Better care. Ottawa, ON: The Royal College of Physicians
and Surgeons of Canada.
Freeman, L. C. (1978/1979). Centrality in social networks conceptual clarification. Social
Networks, 1(3), 215–239. doi:10.1016/0378-8733(78)90021-7
Friedman, M. H., Connell, K. J., Olthoff, A. J., Sinacore , J. M., & Bordage, G. (1998). Thinking
about student thinking: Medical student errors in making a diagnosis [Supplement].
Academic Medicine, 73(10), S19–S21. Retrieved from
http://journals.lww.com/academicmedicine/
Fujita, N., & Teplovs, C. (2009, June). Automating analysis of collaborative discourse:
Identifying idea clusters. Presentation at the International Conference of Computer
Supported Collaborative Learning, Rhodes, Greece.
Gardy, J. L., Johnston, J. C., Ho Sui, S. J., Cook, V. J., Sha, L., Brodkin, E., Rempel, S., . . .
Tang, P. (2011, June). Whole-genome sequencing and social-network analysis of a
tuberculosis outbreak. New England Journal of Medicine, 364,730–739. Retrieved from
http://www.nejm.org/doi/full/10.1056/NEJMoa1003176
Gettier, E. L. (1963). Is justified true belief knowledge? Analysis, 23, pp. 121–123. Retrieved
from http://www.ditext.com/gettier/gettier.html
Glick, M., & Holyoak, K. (1983). Schema induction and analogical transfer. Cognitive
Psychology, 15, 1–38.
Graham, I. D., Logan, J., Harrison, M. B., Straus, S. E., Tetroe, J., Caswell, W., & Robinson, N.
(2006). Lost in knowledge translation: Time for a map? The Journal of Continuing
Education in the Health Professions, 26(1),13–24. doi: 10.1002/chp.47
Granovetter, M. (1983). The strength of weak ties: A network theory revisited. Sociological
Theory, 1, 201–233.
251
Guest, C. B., Regehr, G., & Tiberius, R. G. (2001). The lifelong challenge of expertise. Medical
Education, 35(1), 78–81. doi:10.1111/j.1365-2923.2001.00831.x
Hager, M., Russell, S., & Fletcher, S.W. (Ed.). (2007, November 28–December 1). Continuing
education in the health professions—Improving healthcare through lifelong learning.
Proceedings of a conference sponsored by the Josiah Macy, Jr. Foundation, Bermuda.
New York, NY: Josiah Macy, Jr. Foundation. Retrieved from
http://josiahmacyfoundation.org/docs/macy_pubs/pub_ContEd_inHealthProf.pdf
Hanna, E., Premi, J., & Turnbull, J. (2000). Results of remedial continuing medical education in
dyscompetent physicians. Academic Medicine, 75(2), 174–176. Retrieved from
http://journals.lww.com/academicmedicine
Haythornthwaite, C. (2002). Strong, weak, and latent ties and the impact of new media. The
Information Society, 18, 385–401.
Hewitt, J. (2001). Beyond threaded discourse. International Journal of Educational
Telecommunications, 7(3), 207–221.
Hewitt, J. (2005). Toward an understanding of how threads die in asynchronous computer
conferences. Journal of the Learning Sciences, 14(4), 567–589.
Hewitt, J., & Brett, C. (2007) The relationship between class size and online activity patterns in
asynchronous computer conferencing environments. Computers & Education, 49, 1258–
1271.
Hewitt, J., & Scardamalia, M. (1998). Design principles for distributed knowledge building
processes. Educational Psychology Review, 10(1), 75–96. doi:
10.1023/A:1022810231840
Hewitt, J., & Teplovs, C. (1999). An analysis of growth patterns in computer conferencing
threads [Article 29]. Proceedings of the 1999 conference on Computer Support for
Collaborative Learning. Retrieved from http://portal.acm.org/citation
.cfm?id=1150269&CFID=35298853&CFTOKEN=79054397
Hmelo-Silver, C. E., & Barrows, H. S. (2008). Facilitating collaborative knowledge building.
Cognition and Instruction, 26,(1), 48–94.
Hodges, B. D., Kuper, A., & Reeves, S. (2008). Qualitative research: Discourse analysis. British
Medical Journal, 337, 570–572. doi:10.1134/bmja879
Holyoak, K., & Thagard, P. (1995). Mental leaps: Analogy in creative thought. Cambridge, MA:
MIT Press/Bradford Books.
Hong, H-Y., Scardamalia, M., & Zhang, J. (2010). Knowledge society network: Toward a
dynamic, sustained network for building knowledge. Canadian Journal of Learning and
Technology, 36(1). Retrieved from http://www.cjlt.ca
Jones, G. A., McCarney, P. L., & Skolnik, M. L. (Eds.). (2005). Creating knowledge,
strengthening nations: The changing role of higher education. Toronto, ON: University
of Toronto Press.
252
Kanter, S. (2008, August). Toward better descriptions of innovations. Academic Medicine, 83(8),
703–704.
Keating, N. L., Ayanian, J. Z., Cleary, P. D., & Marsden, P. V. (2007, June). Factors affecting
influential discussions among physicians: A social network analysis of a primary care
practice. Journal of General Internal Medicine, 22(6), 794–796.
Knowles, M. S. (1975). Self-directed learning: A guide for learners and teachers. Englewood
Cliffs, NJ: Prentice Hall Regents.
Kolodner, J. (1992). An introduction to case-based reasoning. Artificial Intelligence Review,
6(1), 3–34. doi:10.1007/BF00155578
Koschmann, T. D., Myers, A. C., Feltovich, P. J., & Barrows, H. S. (1994). Using technology to
assist in realizing effective learning and instruction: A principled approach to the use of
computers in collaborative learning. The Journal of the Learning Sciences, 3(3), 227–
264. doi:10.1207/s15327809jls0303_2
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing
one’s own incompetence lead to inflated self-assessments. Journal of Personality and
Social Psychology, 77(6), 1121–1134.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation.
Cambridge, UK: Cambridge University Press.
Lax, L., Scardamalia, M., Watt-Watson, J., Hunter, J., & Bereiter, C. (2010). Beyond learning
management systems: Designing for interprofessional knowledge building in the health
sciences. Canadian Journal of Learning and Technology, 36(1). Retrieved from
http://www.cjlt.ca
Lax, L., Singh, A., Scardamalia, M., & Librach, L. (2006). Self-assessment for knowledge
building in health care. QWERTY: Interdisciplinary Journal of Technology, Culture and
Education. 1(2), 19–37. Retrieved from http://www.ckbg.org
Lax, L., Watt-Watson, J., Pennefather, P., Hunter, J., & Scardamalia, M. (2002). Project to
examine the cost/effectiveness of e-learning for interfaculty health sciences education.
Unpublished report, funded by The Academic Priority Fund of the Council of Health
Science and Social Work Deans and the University of Toronto Centre for the Study of
Pain, Toronto, ON. Retrieved from http://www.ikit.org/fulltext/Lax_PainWeek
_FinalReport_2002.pdf
Lax, L., Watt-Watson, J., Pennefather, P., Hunter, J. & Scardamalia, M. (2003a, April). E-
learning for strategic clinical management: Advancing beyond identification of students’
conceptions / misconceptions. Poster session presented at the annual meeting of the
Association of Canadian Medical Colleges/Canadian Association for Medical Education,
Quebec City, QC.
Lax, L., Watt-Watson, J., Pennefather, P., Hunter, P., & Scardamalia, M. (2003b). The pain week
e-learning project: An undergraduate interprofessional knowledge building initiative
[Supplement 1]. The Journal of Pain, 4(2), 726.
253
Lee, E. Y. C., Chan, C. K. K., & Van Aalst, J. (2006). Students assessing their own collaborative
knowledge building. International Journal of Computer-Supported Collaborative
Learning, 1, 277–307. Retrieved from http://www.springerlink.com/
Lemieux, M., & Bordage, G. (1992). Propositional versus structural semantic analyses of
medical diagnostic thinking. Cognitive Science, 16, 185–204.
doi:10.1207/s15516709cog1602_2
Lester, H., & Tritter, J. Q. (2001). Medical error: A discussion of the medical construction of
error and suggestions for reforms of medical education to decrease error. Medical
Education, 35(9), 855–861. doi:10.1046/j.1365-2923.2001.01003.x
Liao, S., Alpesh, A., & Rucker, L. (2004). An innovative, longitudinal program to teach residents
about end-of-life care. Academic Medicine, 79(8), 752–757. Retrieved from
http://journals.lww.com/academicmedicine
Ligorio, M. B. (2009). Identity as a product of knowledge building: The role of mediated
dialogue. QWERTY: Interdisciplinary Journal of Technology, Culture and Eduation,
4(1), 33–46. Retrieved from www.ckbg.org/qwerty/indiex.php/qwerty/article/view/46
Ligorio, M. B. (2010). Dialogical relationship between identity and learning. Culture &
Psychology, 16(1), 93–107. doi: 10.1177/1354067X09353206
Lipponen, L., Rahikainen, M., Lallimo, J., & Hakkarainen, K. (2003). Patterns of participation
and discourse in elementary students’ computer-supported collaborative learning.
Learning and Instruction, 13, 487–509.
Mamede, S., & Schmidt, H. G. (2004). The structure of reflective practice in medicine. Medical
Education, 38, 1302–1308. doi:10.1111/j.1365-2929.2004.01917.x
Mandil, S. (1982). The meaning and role of communication networks in health care with regard
to WHO’s goal of “Health for all by the year 2000.” In H. E. Peterson & A. I. Isakkson
(Eds.), Communication networks in health care (pp. 7–42). Amsterdam, the Netherlands:
North-Holland.
Manning, P. R., & DeBakey, L. (2011). Lifelong medical education: Past, present, future. In D.
K. Wentz (Ed.), Continuing medical eduation: Looking back, planning ahead (pp. 17–
20). Hanover, NH: Dartmouth College Press.
Marin, A., & Wellman, B. (2011). Social network analysis: An introduction. In J. Scott & P.
Carrington (Eds.), The Sage handbook of social network analysis (pp. 11–25). London,
UK: Sage.
Marinopoulos, S. S., Dorman, T., Ratanawongsa, N., Wilson, L. M., Ashar, B. H., Magaziner, J.
L., Miller, R. G., . . . Bass, E. B. (2007, January). Effectiveness of continuing medical
education (Evidence Report/Technology Assessment No. 149, Prepared by the Johns
Hopkins Evidence-based Practice Centre, under Contract No. 2900-02-0018, AHRQ
Publication No. 07-E006). Rockville, MD: Agency for Healthcare Research and Quality.
254
Mayer, R. E. (2005). Cognitive theory of multimedia learning. In Mayer, R. E. (Ed.), The
Cambridge handbook of multimedia learning (pp. 31–48). Cambridge: University Press.
Mazmanian, P. E., & Davis, D. A. (2002). Continuing medical education and the physician as
learner: Guide to the evidence. Journal of American Medical Association, 288(9), 1057–
1060. doi:10.1001/jama.288.9.1057
Merriam, S. B. (2001). Qualitative research and case study applications in education. San
Francisco, CA: Jossey-Bass.
Miller, L. (2011). Using social networks to create ACOs (Accountable Care Organizations).
Retreived from http://www.hhnmag.com/hhnmag/HHNDaily/HHNDailyDisplay
.dhtml?id=3230009892
Miller, S. H., Thompson, J. N., Mazmanian, P. E., Aparicio, A., Davis, D. A., Spivey, B. E., &
Kahn, N. B. (2008). Continuing medical education, professional development, and
requirements for medical licensure: A white paper of the Conjoint Committee on
Continuing Medical Education. The Journal of Continuing Education in the Health
Professions, 28(2), 95–98. Retrieved from http://www.jcehp. com/
Mills, S. (1997). Discourse in the new critical idiom. (J. Drakakis, Series Ed.). New York, NY:
Routledge.
Moores, L. K., Dellert, E., Baumann, M. H., & Rosen, M. J. (2009). Effectiveness of continuing
medical education: American College of Chest Physicians Evidence-Based Educational
Guidelines. Chest, 135, 1S–4S. doi:10.1378/chest.08-2511
Morrison, R. S., & Meier, D. E. (2004). Palliative care. The New England Journal of Medicine,
350, 2582–2590. doi:10.1056/NEJMcp035232#t
Moss, J., & Beatty, R. (2010). Knowledge building and mathematics: Shifting the responsibility
for knowledge advancement and engagement. The Canadian Journal of Learning and
Technology, 36(1). Retrieved from http://www.cjlt.ca
Murad, M. H., Coto-Yglesia, F., Varkey, P., Prokop, L., & Murad, A. L. (2010). The
effectiveness of self-directed learning in health professions education: A systematic
review. Medical Education, 44, 1057–1068. doi: 10.1111/j.1365-2923.2010.03750.x
Mylopoulos, M. (2007). Implicit theories of innovating and expertise (Doctoral dissertation).
Retrieved from http://openlibrary.org/works/OL12753654W/Implicit_theories_of
_innovation_and_expertise
Mylopoulos, M., & Scardamalia, M. (2008). Doctors’ perspectives on their innovations in daily
practice: Implications for knowledge creation in health care. Medical Education, 42, 975–
981.
Nendaz, M. R., Junod, A. F., Vu, N. V., & Bordage, G. (1998). Medical students and housestaff:
Partners in learning: Eliciting and displaying diagnostic reasoning during educational
rounds in internal medicine: Who learns from whom? [Supplement]. Academic Medicine,
73(10), S54–S56. Retrieved from http://journals.lww.com/academicmedicine
255
Netminer 3. (n.d.) [Computer software]. Available from http:www.netminer.com
Newell, A. (1980). Reasoning, problem solving and decision processes: The problem space as a
fundamental category. In R. Nickerson (Ed.), Attention and performance VIII (pp. 693–
718). Hillsdale, NJ: Lawrence Erlbaum Associates.
Ng, S. K., & Law, N. W. Y. (2003). Knowledge building in the absence of teacher facilitation. In
Y. S. Chee, N.W. Y. Law, K. T. Lee, & D. Suthers (Eds.), Proceedings of the
International Conference on Computers in Education 2003 (pp. 406–413). Norfolk, VA:
Association for the Advancement of Computing in Education.
Nonaka, I. (1991, November/December). The knowledge-creating company. The Harvard
Business Review, 96–104.
Norman, G. R., & Schmidt, H. G. (1992). The psychological basis of problem-based learning: A
review of the evidence. Academic Medicine, 67(9), 557–565. Retrieved from
http://journals.lww.com/academicmedicine
Organization for Economic Co-operation and Development (OECD). (2007, April). Participative
Web: User-created content. Retrieved from http://www.oecd.org
/dataoecd/57/14/38393115.pdf
Palincsar, A., & Brown, A. (1984). Reciprocal teaching of comprehension-fostering and
comprehension-monitoring activities. Cognition and Instruction, 1, 117–175.
Patel, V. L., Arocha, J. F., & Kaufman, D. R. (1999). Expertise and tacit knowledge in medicine.
In R. J. Sternberg & J. A. Horvath (Eds.), Tacit knowledge in professional practice:
Researcher and practitioner perspectives (pp. 75–120). Mahwah, NJ: Lawrence Erlbaum
Associates.
Patel, V. L., Yoskowitz, N. A., Arocha, J. F., & Shortliffe, E. H. (2009). Cognitive and learning
sciences in biomedical and health instructional design: A review with lessons for
biomedical informatics education. Journal of Biomedical Informatics, 42(1), 176–197.
doi:10.1016/j.jbi.2008.12.002
Pearls™. (n.d.) [Computer software]. Available from http://www.cfpc.ca/Pearls
Pereira, J., Palacios, M., Collin, T., Wedel, R., Galloway, L., Murray, . . . Lockyer, J. (2008).
The impact of a hybrid online and classroom-based course on palliative care
competencies of family medicine residents. Palliative Medicine, 22(8), 1–9.
doi:10.1177/0269216308094561
Petroski, H. (1996). Invention by design. Cambridge, MA: Harvard University Press.
Philip, D. N. (2010). Social network analysis to examine interaction patterns in knowledge
building communities. Canadian Journal of Learning and Technology, 36(1). Retrieved
from http://www.jclt.ca
Pisacane, A. (2008). Rethinking continuing medical education. British Medical Journal, 337,
a973. doi:10.1136/bmj.a973
256
Polanyi, M. (1983). The tacit dimension. Gloucester, MA: Peter Smith.
Popper, K. R. (1972). Objective knowledge: An evolutionary approach. London, UK: Oxford
University Press.
Porter-Williamson, K., von Gunten, C. F., Garman, K., Herbst, L., Bluestein, H. G., & Evans, W.
(2004). Improving knowledge in palliative medicine with a required hospice rotation for
third-year medical students. Academic Medicine, 79(8), 777–782. Retrieved from
http://journals.lww.com/academicmedicine
Posavac, E. J., & Carey, R. G. (2003). Program evaluation: Methods and case studies. Upper
Saddle River, NJ: Prentice Hall.
Price, D., Havens, C., & Bell, M. J. (2011). Continuing professional development and
improvement to meet current and future continuing medical education needs of
physicians. In Wentz, D. K. (Ed.), Continuing medical eduation: Looking back, planning
ahead (pp. 1–14). Hanover, NH: Dartmouth College Press.
Punja, Z. (2007). The role of Knowledge Building in medical education (Unpublished doctoral
dissertation). OISE/UT, Toronto, ON.
Quality End-of-Life Care Coalition of Canada. (2005, September). Framework for a national
strategy on palliative and end-of-life care. Retrieved from http://www.chac.ca/advocacy
/briefs/docs/palliative_brief_qelccc.pdf
Regehr, G., & Eva, K. (2006). Self-assessment, self-direction, and the self-regulating
professional. Clinical Orthopaedics and Related Research, 449, 34–38.
doi:10.1097/01.blo.0000224027.85732.b2
Regehr, G., & Norman, G. (1996). Issues in cognitive psychology: Implications for professional
education. Academic Medicine, 71(9), 988–1001. Retrieved from http://journals.lww.com
/academicmedicine
Romanow, R. C. (2002). Building on values: Commission on the future of health care in Canada
(Final report). Ottawa, ON: Commission on the Future of Health Care in Canada.
Retrieved from www.hc-sc.gc.ca/english/care/romanow/indexl.html
Russell, A. (2005). Progress towards design of a knowledge building community in health care
(Unpublished doctoral dissertation). OISE/UT, Toronto, ON.
Russell, A., Campbell, H., Scardamalia, M., & Bereiter, C. (2002). Fostering interprofessional
practice in rehabilitation and complex continuing care through collaboration and virtual
learning. Retrieved from http:// ikit.org/OLT.html
Saljo, R. (2003). Representational tools and the transformation of learning. In B. Wasson, S.
Ludvigsen, & U. Hoppe (Eds.), Designing for change (pp. 1–2). Retrieved from
www.springer.com/?SGWID=4-102-45-131802-0
Salomon, G. (1994a). Differences in patterns: Studying computer enhanced learning
environments. In E. Vosniadou, E. De Corte, & H. Mandl (Eds.), Technology-based
257
learning environments: Psychological and educational foundations (pp. 79–88). Berlin,
Germany: Springer-Verlag
Salomon, G. (1994b). Interaction of media, cognition, and learning. Hillsdale, NJ: Lawrence
Erlbaum Associates.
Sargeant, J., Mann, K., van der Vleuten, C., & Metsemakers, J. (2008). Directed self-assessment:
Practice and feedback within a social context. The Journal of Continuing Education in
the Health Professions, 28(1), 47–54. Retrieved from http://www.jcehp. com
Sawyer, R. K. (Ed.), (2006). The Cambridge handbook of the learning sciences. Cambridge, UK:
Cambridge University Press.
Scardamalia, M. (1999). Moving ideas to the centre. In L. Harasim (Ed.), Wisdom and wizardry:
Celebrating the pioneers of online education (pp. 14–15). Vancouver, BC: Telelearning.
Scardamalia, M. (2002). Collective cognitive responsibility for the advancement of knowledge.
In B. Smith (Ed.), Liberal education in a knowledge society (pp. 67–98). Chicago, IL:
Open Court.
Scardamalia, M. (2003a). Knowledge building. (Advances beyond CSILE). [Supplement 3,
“Learning Technology Innovation in Canada”]. Journal of Distance Education, 17, 10–
14.
Scardamalia, M. (2003b). Knowledge society network (KSN): Toward an expert society for
democratizing knowledge [Supplement 3, “Learning Technology Innovation in Canada”].
Journal of Distance Education, 17, 63–66.
Scardamalia, M. (2004a). CSILE/Knowledge forum. In A. Kovalchick, & K. Dawson (Eds.),
Educational technology: An encyclopedia (pp. 183–192). Santa Barbara, CA: ABC-CILO
Scardamalia, M. (2004b). Instruction, learning, and knowledge building: Harnessing theory,
design, and innovation dynamics. Educational Technology, 44(3), 30–33.
Scardamalia, M., & Bereiter, C. (1991). Higher levels of agency for children in knowledge
building: A challenge for the design of new knowledge media. The Journal of the
Learning Sciences, 1(1), 37–68. doi:10.1207/s15327809jls0101_3
Scardamalia, M., & Bereiter, C. (1994). Computer support for knowledge-building communities.
The Journal of the Learning Sciences, 3(3), 265–283. doi:10.1207/s15327809jls0303_3
Scardamalia, M., & Bereiter, C. (1996). Student communities for the advancement of
knowledge. Communications of the ACM, 39(4), 36–37.
Scardamalia, M., & Bereiter, C. (2003a). Knowledge building. In J. W. Guthrie (Ed.),
Encyclopedia of Education (2nd ed., pp. 1370–1373). New York, NY: Macmillan
Reference Library.
Scardamalia, M., & Bereiter, C. (2003b). Knowledge building environments: Extending the
limits of the possible in education and knowledge work. In A. DiStefano, K. E.
258
Rudestam, & R. Silverman (Eds.), Encyclopedia of distributed learning (pp. 269–272).
Thousand Oaks, CA: Sage.
Scardamalia, M. & Bereiter, C. (2005). Does education for the knowledge age need a new
science? European Journal of School Psychology, 3(1), 21–40.
Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and
technology. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences
(pp. 97–115). New York, NY: Cambridge University Press.
Scardamalia, M., & Bereiter, C. (2010). A brief history of knowledge building. Canadian
Journal of Learning and Technology, 36(1). Retrieved from http://www.cjlt.ca
Scardamalia, M., Bereiter, C., McLean, R., Swallow, J., & Woodruff, E. (1989). Computer-
supported intentional learning environments. Journal for Educational Computing
Research, 5(1), 51–68. doi:10.2190/CYXD-6XG4-UFN5-YFBO
Scardamalia, M., Bransford, J., Kozma, R., & Quellmalz, E. (in press). New assessments and
environments for knowledge building. In P. Griffin, B. McGaw, & E. Care (Eds.),
Assessment and learning of 21st Century Skills (pp. 231–301). New York, NY: Springer.
Retrieved from http://atc21s.org/default.aspx
Schmidt, H. G., & Boshuizen, H. P. A. (1992). Encapsulation of biomedical knowledge. In D. A.
Evans & V. L. Patel (Eds.), Advanced models of cognition for medical training and
practice (pp. 265–282). Berlin, Germany: Springer-Verlag.
Schmidt, H. G., Norman, G. R., & Boshuizen, H. P. A. (1990). A cognitive perspective on
medical expertise: Theory and implications. Academic Medicine, 65(10), 611–621.
Erratum published April1992, in Academic Medicine, 67(4), 287. Retrieved from
http://journals.lww.com/academicmedicine
Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York,
NY: Basic Books.
Schön, D. A. (1987). Educating the reflective practitioner: Toward a new design for teaching
and learning in the professions. San Francisco, CA: Jossey-Bass.
Schuwirth, L., & van der Vleuten, C. (2004). Merging views on assessment. Medical Education,
38, 1208–1210. doi:10.1111/j.1365-2929.2004.02055.x
Scott, J. (2001). Power. Cambridge, UK: Polity Press.
Scott, J. (2009). Social network analysis: A handbook (2nd ed.). Thousand Oaks, CA: Sage.
Scott, J., & Carrington, P. (Eds.). (2011). The Sage handbook of social network analysis.
London, UK: Sage.
Self Learning™. (n.d.) [Computer software]. Available from http://www.cfpc.ca/sli
259
Sha, L., & van Alst, J. (2003, April 21–25). An application of social network analysis to
knowledge building. Proceedings of the Annual Meeting of the American Educational
Research Association. Chicago, IL.
Sha, L., Teplovs, C., & van Aalst, J. (2010, June). A visualization of group cognition: Semantic
network analysis of a CSCL community. In K. Gomez, L. Lyons, & J. Radinsky (Eds.),
Learning in the disciplines: Proceedings of the Ninth International Conference of the
Learning Sciences (Vol. 1, pp. 929–936). Chicago, IL: International Society of the
Learning Sciences.
Sibbald, D. J. (2009). Potential for knowledge building in large size pharmacy classrooms
(Doctoral dissertation). Retrieved from Library and Archives Canada. (ISBN: 978-0-494-
61091-6)
Silver, I., Campbell, C., Marlow, B., & Sargeant, J. (2008). Self-assessment and continuing
professional development: The Canadian perspective. The Journal of Continuing
Education in the Health Professions, 28(1), 25–31. Retrieved from http://www.jcehp
.com/
Silverdale, N., & Katz, J. (2005). The impact of a distance learning death and dying course: An
analysis of student self-reported changes. Nurse Education Today, 25(7), 509–518.
doi:10.1016/j.nedt.2005.05.004
Singer, P. A., & Bowman, K. W. (2002). Quality end-of-life care: A global perspective.
Palliative Care, 1(4). doi: 10.1186/1472-684X-1-4
Singer, P. A., Martin, D. K., Kelner, M. (1999). Quality end of life care: Patients’ perspectives.
Journal of the American Medical Association, 281, 163–168.
Spiro, R. J., Coulson, R. L., Feltovich, P. J., & Anderson, D. (1988). Cognitive flexibility theory:
Advanced knowledge acquisition in ill-structured domains. In V. Patel (Ed.), Proceedings
of the 10th Annual Conference of the Cognitive Science Society (pp. 433–439). Hillsdale,
NJ: Lawrence Erlbaum Associates.
Spiro, R. J., Feltovich, P. J., Jacobson, M. J., & Coulson, R. L. (1991). Cognitive flexibility,
constructivism, and hypertext: Random access instruction for advanced knowledge
acquisition in ill-structured domains. Educational Technology, 31(5), 24–33.
Spiro, R. J., & Jehng, J. C. (1991). Cognitive flexibility and hypertext: Theory and technology
for the nonlinear and multidimensional traversal of complex subject matter. In D. Nix &
R. Spiro (Eds.), Cognition, education, and multimedia: Exploring ideas in high
technology (pp. 163–173). Mahwah, NJ: Lawrence Erlbaum Associates.
Stokes, D. (1997). Pasteur’s quadrant basic science and technological innovation. Washington,
DC: The Brookings Institution.
Sullivan, A. M., Warren, A. G., Lakoma, M. D., Liaw, K. R., Hwang, D., & Block, S. D. (2004).
End-of-life care in the curriculum: A national study of medical education deans
[Research report]. Academic Medicine, 79(8), 760–768. Retrieved from http://journals
.lww.com/academicmedicine
260
Sutherland, S. (2011, January 27). Education rounds. Retrieved from http://www.med.uottawa.ca
/aime/eng/education_rounds.html
Sutherland, S., Moineau, G., Leddy, J., Cohen, E., Hamstra, S., & Regehr, G. (2011, April).
Understanding the utility of social network analysis for the medical trainee. AIME 4th
Annual Medical Education Day (p. 5). University of Ottawa, Academy for Innovation in
Medical Education. Retrieved from http://www.med.uottawa.ca/aime/assets
/documents/aime_day_4_abstracts.pdf
Tabak, I., & Baumgartner, E. (2004). The teacher as partner: Exploring participant structures,
symmetry, and identity work in scaffolding. Cognition and Instruction, 22(4), 393–
429.doi: 10.1207/s1532690Xci2204_2
Teplovs, C. (2010). Visualization of knowledge spaces to enable concurrent, embedded, and
transformative input to knowledge building processes (Unpublished doctoral
dissertation). OISE/UT, Toronto, ON.
Teplovs, C., & Fujita, N. (2009). Determining curricular coverage of student contributions to an
online discourse environment: Using latent semantic analysis to construct differential
term clouds. In C. O’Malley, D. Suthers, P. Reimann, & A. Dimitracopoulou (Eds.),
Computer supported collaborative learning practices (pp.165–167). Rhodes, Greece:
International Society of the Learning Sciences.
Teplovs, C., & Scardamalia, M. (2007). Visualizations for knowledge building assessment. Paper
presented at the AgileViz workshop, Computer Support for Collaborative Learning
Conference. Retrieved from http://chris.ikit.org/agilevizcscl-teplovs-scardamalia.pdf
Thagard, P. (1997). Medical analogies: Why and how. In P. Langley & M. Shafto (Eds.),
Proceedings of the Nineteenth Annual Conference of the Cognitive Science Society (pp.
739–744). Mahwah, NJ: Lawrence Erlbaum Associates.
Thagard, P. (2000). Coherence in thought and action. Cambridge, MA: MIT Press.
Thagard, P. (2007, January). Coherence, truth, and the development of scientific knowledge.
Philosophy of Science, 74, 28–47. Retrieved from http://cogsci.uwaterloo.ca/Articles
/coherence.truth.pos.2007.pdf
doi:10.1086/520941
Thagard, P., & Shelley, C. (1997) Abductive reasoning: Logic, visual thinking, and coherence.
In: M.-L. Dalla Chiara et al (eds), Logic and scientific methods (pp. 413–427). Dordrecht,
the Netherlands: Kluwer Academic. Retrieved from http://cogprints.org/671/1
/FAbductive.html
University of Toronto, Faculty of Medicine, Office of Continuing Education & Professional
Development. (2011). Strategic academic plan 2011–2016. Retrieved from http://www
.facmed.utoronto.ca/Page14.aspx
Valente, T. W. (2010). Social networks and health: Models, methods and applications. Oxford,
UK: Oxford University Press.
261
Viilo, M., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2011). Supporting the technology-
enhanced collaborative inquiry and design project: A teacher’s reflections on practices.
Teachers and Teaching Theory and Practice, 17, 51–72.
Von Hippel, E. (2005). Democratizing innovation [E-book, licensed under a Creative Commons
License]. Retrieved from http://www.mit.edu/people/evhippel/democ1.htm
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes.
Cambridge, MA: Harvard University Press.
Walton, J. M., & Oswald, A. (2011, May 7–11). Look who’s talking: Patterns of interaction in
problem-based learning. Scholarship in Medical Education, Canadian Conference in
Medical Education, Toronto, ON. Retrieved from http://www.mededconference.ca
/documents/Program2011_eng_WEB.pdf
Wang, Y., & Li, X. (2007). Social network analysis of interaction in online learning
communities. Seventh IEEE International Conference on Advanced Learning
Teachnologies, Niigata, Japan.
Wang, L. (2010). How social network position relates to knowledge buildng in online learning
communities. Frontiers of Education in China, 5(1), 4–25. doi: 10.1007/s11516-010-
000304
Wasserman, S., & Faust, K. (1994). Social network analysis: methods and applications.
Cambridge, UK: Cambridge University Press.
Watt-Watson, J., Hunter, J., Pennefather, P., Librach, L., Raman-Wilms, L., Schreiber, M.,
. . . Salter, M. (2004). An integrated undergraduate pain curriculum based on IASP
curricula, for six health sciences facilities. Pain, 110, 140–148.
Wear, D. (2002). “Face-to-face with it”: Medical students’ narratives about their end-of-life
education. Academic Medicine, 77(4), 271–277. Retrieved from http://journals.lww
.com/academicmedicine
Weissman, D. E., & Block, S. D. (2002). ACGME requirements for end-of-life training in
selected residency and fellowship programs. Academic Medicine, 77(4), 299–304.
Retrieved from http//journals.lww.com/academicmedicine
Wellman, B. (1983). Network analysis: Some basic principles. Sociological Theory, 1, 155–200.
Retrieved from http://www.jstor.org/stable/202050
Wellman, B. (1988). Structural analysis: From method and metaphor to theory and substance. In
B. Wellman & S. D. Berkowitz (Eds.), Social structures: A network approach (pp. 19–
61). Cambridge, UK: Cambridge University Press.
Wenger, E. C. (1998). Communities of practice: Learning, meaning, and identity. Cambridge,
UK: Cambridge University Press.
Wentz, D. K. (Ed.) Continuing medical education: Looking back, planning ahead. Hanover, NH:
Dartmouth College Press.
262
Whitehead, A. N. (1933). Adventures of ideas. New York, NY: The Free Press.
Whitehead, A. N. (1967). The aims of education and other essays. New York, NY: The Free
Press.
World Health Organization (WHO). (1990). Cancer pain relief and palliative care (World
Health Organization Technical Report, Series 804). Geneva, Switzerland: Author.
Wood, E. W., Meekin, S. A., Fins, J. J., & Fleischman, A. R. (2004, April). Enhancing palliative
care education in medical school curricula: Implementation of the palliative education
assessment tool. Academic Medicine, 77(4), 285–291.
Woollard, R. (2011). A history of the committee on the accreditation of continuing medical
education—Canada. In D. K. Wentz (Ed.), Continuing medical education: Looking back,
planning ahead (pp. 218–226). Hanover, NH: Dartmouth College Press.
Zhang, J., Scardamalia, M., Lamon, M., Messina, R., & Reeve, R. (2007). Socio-cognitive
dynamics of knowledge building in the work of 9- and 10-year-olds. (2007, April).
Educational Technology, Research and Development, 55(2),117–145. doi:
10.1007/s11423-006-9019-0
Zhang, J., Scardamalia, M., Lamon, M., Messina, R., Reeve, R., & Teplovs, C. (2005, April 11–
15). How does knowledge work in classrooms: Knowledge dynamics in online knowledge
building environment. Paper presented at the American Educational Research Association
annual meeting, Montreal, QC.
Zhang, J., Scardamalia, M., Reeve, R., & Messina, R. (2009). Designs for collective cognitive
responsibility in knowledge-building communities. The Journal of the Learning Sciences,
18(1), 7–44. doi: 10.1080/10508400802581676
267
APPENDIX C
End-of-Life Care Distance Education Program Modules in Knowledge Forum®
and Multimedia Case Note
268
APPENDIX D
Pain Pre-/Posttest Item Analysis 2004/2005
EOLC Pain Pre-/Posttest
2004-05 Item Analysis (N = 9)
Question Objective Sig Sig Mean Mean
2-tailed 1-tailed Pre Post Diff
q1 1 1.000 0.500 89 100 11
q2 1 na na 100 100 0
q3 3 na na 100 100 0
q4 3 0.625 0.313 56 78 22
q5 2 0.250 0.125 33 67 33
q6 2 0.500 0.250 67 89 22
q7 2 1.000 0.500 0 11 11
q8_1 2 1.000 0.500 22 33 11
q8_2 2 0.125 0.063 56 100 44
q8_4 2 1.000 0.500 33 44 11
q8_5 2 1.000 0.500 56 56 0
q8_7 2 1.000 0.500 89 100 11
q9_1 2 0.375 0.188 11 44 33
q9_2 2 0.063 0.031 33 89 56
q9_4 2 1.000 0.500 11 22 11
q9_5 2 1.000 0.500 44 56 11
q10_1 2 na na 0 0 0
q10_2 2 1.000 0.500 22 11 -11
q10_4 2 na na 0 0 0
q10_5 2 0.250 0.125 33 67 33
q11_1 2 0.625 0.313 33 11 -22
q11_2 2 0.016 0.008 78 0 -78
q11_4 2 1.000 0.500 11 0 -11
q11_5 2 1.000 0.500 56 56 0
q11_7 2 1.000 0.500 89 100 11
q12 2 1.000 0.500 22 33 11
q13 4 0.250 0.125 11 44 33
q14 4 0.125 0.063 56 100 44
q15 1 1.000 0.500 89 100 11
q16 1 0.500 0.250 67 89 22
q17 1 na na 100 100 0
269
q18 2 1.000 0.500 22 11 -11
q19 2 1.000 0.500 89 89 0
q20 2 1.000 0.500 78 78 0
q21 3 0.250 0.125 33 67 33
q22_1 3 1.000 0.500 67 78 11
q22_2 3 1.000 0.500 67 78 11
q22_3 3 1.000 0.500 89 89 0
q22_4 3 1.000 0.500 67 67 0
q22_5 3 1.000 0.500 67 78 11
q22_6 3 1.000 0.500 67 78 11
q22_7 3 1.000 0.500 56 67 11
q23 4 0.500 0.250 78 100 22
q24 4 0.250 0.125 67 100 33
q25 1 na na 100 100 0
q26 4 na na 0 0 0
q27 4 1.000 0.500 67 78 11
q28 4 0.500 0.250 78 100 22
q29 4 0.250 0.125 67 100 33
q30 4 1.000 0.500 78 89 11
q31_1 3 1.000 0.500 89 100 11
q31_2 3 1.000 0.500 89 100 11
q31_3 3 0.500 0.250 78 100 22
q31_4 3 0.625 0.313 67 89 22
q31_5 3 1.000 0.500 89 89 0
q31_6 3 na na 100 100 0
q32_1 4 na na 100 100 0
q32_2 4 na na 100 100 0
q32_3 4 1.000 0.500 89 100 11
q32_4 4 na na 100 100 0
q32_5 4 1.000 0.500 89 100 11
q32_6 4 na na 100 100 0
272
APPENDIX E
Pain Pre-/Posttest Item Analysis 2005/2006
EOLC Pain Pre-/Posttest 2005-06
Item Analysis (N=11)
Question
Objectives
Pre-test
(%)
Post-test
(%)
Exact Sig.
(2-tailed) 1-tailed
Post-Pre
% Diff.
Q1 1 73 91 .500 .250 18
Q2 1 73 82 1.000 .500 9
Q3 3 100 100 na na 0
Q4 3 91 91 1.000 .500 0
Q5 2 18 64 .063 .031 45
Q6_1 2 100 100 na na 0
Q6_2 2 100 100 na na 0
Q6_3 2 100 100 na na 0
Q6_4 2 82 100 .500 .250 18
Q7 2 91 82 1.000 .500 -9
Q8 2 18 36 .500 .250 18
Q9_1 2 82 82 1.000 .500 0
Q9_2 2 91 100 1.000 .500 9
Q9_3 2 55 64 1.000 .500 9
Q10_1 2 73 73 1.000 .500 0
Q10_2 2 91 100 1.000 .500 9
Q10_3 2 82 73 1.000 .500 -9
Q11_1 2 18 36 .625 .313 18
Q11_2 2 73 73 1.000 .500 0
Q11_3 2 36 55 .500 .250 18
Q12_1 2 82 91 1.000 .500 9
Q12_2 2 91 91 1.000 .500 0
Q12_3 2 45 64 .625 .313 18
Q12_4 2 91 100 1.000 .500 9
Q13 2 55 82 .250 .125 27
Q14 4 0 9 1.000 .500 9
Q15 4 64 91 .250 .125 27
Q16 1 91 91 1.000 .500 0
Q17 1 100 91 1.000 .500 -9
Q18 1 100 100 na na 0
Q19 2 64 91 .250 .125 27
Q20 2 100 91 1.000 .500 -9
273
Q21 2 91 91 1.000 .500 0
Q22 3 64 82 .625 .313 18
Q23_1 3 91 100 1.000 .500 9
Q23_2 3 91 73 .625 .313 -18
Q23_3 3 82 82 1.000 .500 0
Q23_4 3 64 82 .625 .313 18
Q23_5 3 91 100 1.000 .500 9
Q23_6 3 82 100 .500 .250 18
Q23_7 3 82 100 .500 .250 18
Q24 4 82 100 .500 .250 18
Q25 4 91 100 1.000 .500 9
Q26 1 100 100 na na 0
Q27 4 82 100 .500 .250 18
Q28 4 91 91 1.000 .500 0
Q29 4 91 100 1.000 .500 9
Q30 4 82 91 1.000 .500 9
Q31 4 100 100 na na 0
Q32_1 3 100 100 na na 0
Q32_2 3 100 100 na na 0
Q32_3 3 100 91 1.000 .500 -9
Q32_4 3 82 91 1.000 .500 9
Q32_5 3 82 100 .500 .250 18
Q32_6 3 100 100 na na 0
Q33_1 2 100 100 na na 0
Q33_2 2 100 100 na na 0
Q33_3 2 91 91 1.000 .500 0
Q33_4 2 100 91 1.000 .500 -9
Q33_5 2 91 91 1.000 .500 0
Q33_6 2 100 100 na na 0
Top Related