Complexity and the science of implementation in health IT-Knowledge gaps and future visions

I

CI

Pa

b

c

d

e

f

g

a

A

R

A

K

I

C

I

i

C

i

1h

ARTICLE IN PRESSJB-3044; No. of Pages 11

i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 4 ) xxx–xxx

j ourna l h omepage: www.i jmi journa l .com

omplexity and the science of implementation in healthT—Knowledge gaps and future visions

atricia A. Abbotta,b,∗, Joanne Foster c, Heimar F. Marind, Patricia C. Dykese,f,g

Division of Nursing Business & Health Systems, University of Michigan School of Nursing, Ann Arbor, MI, USAOffice of Global Outreach, University of Michigan School of Nursing, Ann Arbor, MI, USASchool of Nursing, Queensland University of Technology, Brisbane, AustraliaSchool of Nursing, Universidade Federal de São Paulo, BrazilCenter for Patient Safety, Research & Practice, Brigham and Women’s Hospital, MA, USACenter for Nursing Excellence, Brigham and Women’s Hospital, MA, USAHarvard Medical School, Boston, MA, USA

r t i c l e i n f o

rticle history:

eceived 1 July 2013

ccepted 24 October 2013

eywords:

mplementation science

omplexity

mplementation of health

nformation technology

onsolidated framework for

mplementation research

a b s t r a c t

Objectives: The intent of this paper is in the examination of health IT implementation pro-

cesses – the barriers to and facilitators of successful implementation, identification of a

beginning set of implementation best practices, the identification of gaps in the health IT

implementation body of knowledge, and recommendations for future study and application.

Methods: A literature review resulted in the identification of six health IT related imple-

mentation best practices which were subsequently debated and clarified by participants

attending the NI2012 Research Post Conference held in Montreal in the summer of 2012.

Using the framework for implementation research (CFIR) to guide their application, the six

best practices were applied to two distinct health IT implementation studies to assess their

applicability.

Results: Assessing the implementation processes from two markedly diverse settings illus-

trated both the challenges and potentials of using standardized implementation processes.

In support of what was discovered in the review of the literature, “one size fits all” in health

IT implementation is a fallacy, particularly when global diversity is added into the mix. At

the same time, several frameworks show promise for use as “scaffolding” to begin to assess

best practices, their distinct dimensions, and their applicability for use.

Conclusions: Health IT innovations, regardless of the implementation setting, requires a close

assessment of many dimensions. While there is no “one size fits all”, there are commonali-

ties and best practices that can be blended, adapted, and utilized to improve the process of

implementation. This paper examines health IT implementation processes and identifies a

beginning set of implementation best practices, which could begin to address gaps in the

health IT implementation body of knowledge.

Please cite this article in press as: P.A. Abbott, et al., Complexity and the svisions, Int. J. Med. Inform. (2014), http://dx.doi.org/10.1016/j.ijmedinf.2013

∗ Corresponding author at: Division of Nursing Business & Health SystemE-mail address: [email protected] (P.A. Abbott).

386-5056/$ – see front matter © 2013 Elsevier Ireland Ltd. All rights resttp://dx.doi.org/10.1016/j.ijmedinf.2013.10.009

© 2013 Elsevier Ireland Ltd. All rights reserved.

cience of implementation in health IT—Knowledge gaps and future.10.009

s, University of Michigan School of Nursing, Ann Arbor, MI, USA.

erved.

dx.doi.org/10.1016/j.ijmedinf.2013.10.009


www.ijmijournal.com

mailto:[email protected]


ARTICLE IN PRESSIJB-3044; No. of Pages 11

c a l i
2 i n t e r n a t i o n a l j o u r n a l o f m e d i
1. Introduction and review of the literature

Deploying new technology and practice innovations in com-plex healthcare environments is challenging, particularlywhen the innovation is disruptive to established structuresand workflow. Electronic health record systems (EHRS) and/orother types of health information technology (IT) are con-sidered disruptive technologies, and their integration intopractice has been slow and problematic. Even in light ofnumerous reports of the benefits of EHRS, when evidencemeets the realities of practice [1], successful deployment andadoption can be threatened. A plethora of studies point tosystems that work well in controlled situations or in the lab,but fail miserably upon implementation in a naturalistic oralternative environment [2–5].

The seeming disconnect between the evidence and thereality post-implementation raises the question of where theproblem lies. Is the actual technology so faulty that its deploy-ment and use results in harm – or is it in the manner ofimplementation that gives rise to untoward effects? In reality,it is probably a combination of the two, which opens numerousavenues for focused inquiry.

A growing number of researchers are convinced thatachieving levels of success anticipated from EHRS is depend-ent not only upon the features and functions of the systemitself – but also in the manner in which the system is imple-mented [6,23]. The challenge is that implementing systemsinto complex and chaotic healthcare environments is diffi-cult and disruptive – and the evidence on how to improvethe process is low. We suggest that complexity theory andthe methods and models from implementation science can(and should) be used to reframe the thinking about health ITimplementation.

Numerous studies are found in the healthcare literatureregarding the application of implementation science tech-niques in general; however there is a paucity of literaturerelated to the use of these approaches specifically in health ITimplementations. This has resulted in a poor understandingof the impact that the process of implementation may have onthe outcomes of a health IT intervention. Consequently, manystudies regarding EHRs result in reports of failure to achievesignificance or effect meaningful change or, more worrisome,show declines in quality, safety, and efficiency after the imple-mentation of an EHRS. Failure to pay attention to the threatsrelated to the process of implementation results in findingsthat do not tell the whole story. The infamous article by Hanet al. [2] in 2005 is a case in point. While there was an increasein infant mortality after an EHR implementation and therewere distinct issues with the technology, much of the causa-tion for the negative outcomes fell squarely on an extremelypoor implementation process [19].

Eight years after the Han et al. article however, there is stillscant attention being paid to this critical aspect of implemen-tation science. The inability to capture, represent, and reuselessons learned in the implementation process dooms us to


repeat them. The issue is further compounded when takeninto the international context. In addition to understandingcomplex processes present in an organization, the impactof cultural differences can markedly affect outcomes. Cross

n f o r m a t i c s x x x ( 2 0 1 4 ) xxx–xxx

cultural study, workflow inquiry, and adaptation to situation isrequired. The stakes are high; the use of EHRs will continue toexpand as global resources continue to contract. The impetusto understand, deploy and capitalize upon effective productsand processes is marked – particularly as related to health IT.

The focus of this paper is in the examination of health ITimplementation processes – the barriers to and facilitators ofsuccessful implementation, identification of a beginning setof implementation best practices, the identification of gaps inthe health IT implementation body of knowledge, and recom-mendations for future study and application.

1.1. Implementation science, complexity science, andintersections with health IT

Health IT implementation is strongly influenced by the con-text in which it occurs. Clinical environments are complex,unpredictable, and replete with convoluted and highly inter-dependent relationships. Variation is the norm, and nonlinearand novel responses are commonplace. Hospitals and clin-ics fall therefore into the category of a complex and adaptivesystem (CAS).

As posited by Leykum and colleagues [7], implementationsof any sort in a CAS requires creative and critical thinking,acceptance that each system is unique, complex, and contin-ually changing, and understanding that methods that workin one organization or location may fail in another. In addi-tion, changes over time and the influence of the interventionitself on the environment will require a continual adaptationof the methods and models used to study the impact of theintervention [23]. In short, applying traditional approachesto the evaluation of HIT implementation is insufficient forgaining the level of understanding required in such complexenvironments [7]. As advocated by many [7–10,23] a deeperappreciation for the tenets of complexity science and applica-tion of implementation science principles is advised to trulyunderstand the processes and maximize the efficiency of HITimplementations in CAS.

Complexity science approaches are focused upon the iden-tification of interacting elements, an understanding of theirunpredictability, interdependencies and interactions, and thepotential impact that these elements may have on an out-come of interest [11]. Implementation science is the “studyof methods that promote the integration of research findingsand evidence into health care policy and practice. It seeks tounderstand the behavior of healthcare professionals and otherstakeholders as a key variable in the sustainable uptake, adop-tion, and implementation of evidence-based interventions”[12]. Combining complexity and implementation science prin-ciples and applying them to the study of the implementationof health IT therefore would require the knowledge that theunique and continually shifting characteristics of the health-care environment will impact the methods and models usedin a study. Moreover, the understanding that adaptation ofa set approach based on context and evolution of the envi-

cience of implementation in health IT—Knowledge gaps and future3.10.009

ronment over time is the norm. We believe that applyingcomplexity and implementation sciences to health IT imple-mentation will facilitate a deeper understanding of processbarriers and facilitators, enabling the creation of a knowledge




a l i n f o r m a t i c s x x x ( 2 0 1 4 ) xxx–xxx 3

bi

imrcftodagiiac

sr(tt[ScfasItifeaai

ibkTaiEcrd

na“atsvt6sp

Fig. 1 – Chaudoir et al. multilevel framework for

i n t e r n a t i o n a l j o u r n a l o f m e d i c

ase of transferrable best practices for successful health ITmplementation.

An additional challenge to gaining the needed understand-ng and converting knowledge into replicable implementation

ethods however lies in the comprehensive yet non-edundant identification of the constructs that representharacteristics of interest and the use of a scaffold thatacilitates their application. Numerous models, frameworks,axonomies, and methods exist [13], but the standardizationf terms and the description of approaches are poor. Trying toetermine the meaning of ill-defined constructs, discern theirppropriateness for a given situation at varying levels (patient,roup, organizational, and policy [14]), and then applying themn a systematic way is difficult. The use of a framework to guidemplementation studies is another critical consideration as

knowledge base for studies of health IT implementation isonceptualized.

A variety of frameworks for implementation research existuch as the Dynamic Adaptation Process (DAP) [24], the Explo-ation, Preparation, Implementation, and Sustainment ModelEPIS) [25], variants of Social-Technical Frameworks [26–29],he Veterans Affairs QUERI [21], Reach, Effectiveness, Adop-ion, Implementation, and Maintenance Framework (RE-AIM)22], Institute for Healthcare Improvement’s Framework forpread [30], Participatory Design [32], and Normalization Pro-ess Theory (NPT) based frameworks [20]. The consolidatedramework for implementation research (CFIR), one of severaldditional frameworks evident in the literature, is one thateems to be practical and well suited for application in healthT implementation studies [10]. A recent and intriguing studyhat utilized the CFIR specifically discusses its use in bridgingnformatics and implementation science [10]. The recency andocus upon EHRS of the CFIR model by Richardson et al. influ-nced our decision to devote additional attention to it in thisrticle. A brief overview of the CFIR follows, accompanied with

suggested amendment to its terminology for application inmplementation of health IT.

The CFIR was derived from a meta-analysis of existingmplementation theories, and designed to not only clarify,ut to create a framework for “creating an implementationnowledge base across multiple studies and settings” [14].he CFIR is composed of five domains: intervention char-cteristics, outer-setting, inner-setting, characteristics of thendividual involved, and the process of implementation [15].ach of the five domains contain defining constructs and asso-iated methods for measuring the constructs. Readers areeferred to the work of Damschroder et al. [14] for detailedescriptions of the original CFIR and its associated constructs.

To address the need for further refinement of the origi-al CFIR, Chaudoir, Dugan and Barr [16] in 2013 added andditional defining characteristic to the fourth domain ofcharacteristics of individual involved” [15]. The authors cited

concern that the original CFIR (and its association withhe individual only within the context of an organizationaltructure) did not allow for the consideration of patient-levelariables critical to the success of patient-centric interven-


ions [16]. Chaudoir et al. also identifies a beneficial set of2 available measures that “can be utilized to assess con-tructs representing structural-, organizational-, provider-,atient-, and innovation-level factors – factors that are each

implementation research [16].

hypothesized to affect implementation outcomes.” These 62measures may be of value to individuals interested in imple-mentation science applied to HIT. This model is representedin Fig. 1.

The enhanced CFIR suggested by Chaudoir et al. [16] withthe addition of the patient-level variable consideration isvaluable and particularly apropos for use with health IT imple-mentation studies as patient-facing health IT interventionsaccelerate. However, in applying the adapted CFIR to the eval-uation of health IT implementation, we believe that a slightchange in the Chaudoir et al. terminology from the termpatient-level to user-level is warranted. While the suggestion ofthe term user-level that we suggest may appear to return toDamschroder et al. [14] initial CFIR terminology of “individ-ual characteristics” the original and important differentiationraised by Chaudoir et al. remains. The original use of the termof “individual characteristics” was used to describe the inter-relationships and impact of individual characteristics at theorganizational level, which is an unnecessary constraint.

As an overarching term, “users” would more holisticallyrepresent a myriad of individuals, outside of the organizationaldimension, whose characteristics can exert influence on theimplementation and adoption of health IT, such as fami-lies, communities and social norms. In example, the use oftelehealth applications implemented in the home may beinfluenced by family dynamics, cultural beliefs, political lean-ings, or perception – regardless of a patient’s opinion. Thesphere of influence is larger than at the patient-level. Thispoint is further supported by a recent study of habituationand implementation research [17] which makes the point thathumans are humans – regardless of whether they are cli-nicians, families, communities, or patients. Internal beliefs,habituation, and motivations, external to the organizationalcontext, will influence intentions, impacting implementationon the level of the user (who may or may not be a patient)and that these influences carry forth into behaviors regard-less of setting. It would seem therefore, that expanding theterminology suggested by Chaudoir et al. [16] from “patient”


to “user”, is prudent for use of the CFIR in health IT implemen-tation studies. Because this adaptation of the terminology issuggestive in nature and has yet to be validated, we advise the




c a l i
use of the Chaudoir conceptual framework as published andrepresented in Fig. 1. Finally, while the focus has been upon theCFIR in this discussion, it should not imply that other modelsand frameworks are not applicable. As implementation sci-ence and its application to EHRS interventions continues todevelop, additional uses, modifications, and new models areexpected and encouraged.

1.2. Summing up the literature

The fundamentals that have been provided above are intendedto encourage a reframing of the manner in which health ITimplementation studies are conceptualized and undertaken,and the authors are cognizant of the fact that this area ofinquiry requires much more work. Space constraints andpaper intent restrict the in-depth review and debate necessary.It is hoped however that informaticians and others interestedin health IT implementation and adoption will begin to debateand consider the use of complexity theory, implementationscience, and implementation frameworks in efforts of thisnature. In this paper, we begin to consider such work, andprovide the results and discussion of our efforts to date below.

2. Methods and approach: Best practices inHIT implementation – beginning steps

Since implementation science in general has been consid-ered by many to be in its infancy, the lack of literature foundthat discusses implementation science within the context ofhealth IT is not surprising, yet is presented a challenge for theidentification and application of best practices in HIT imple-mentation. As implementation science continues to evolveand its application in health IT interventions increases –there will be a concomitant growth of the knowledge baseof proven models, methods and terminologies. In the interimhowever, several aspects of complexity theory and implemen-tation science can be borrowed and used as a starting pointfor considering evidence-based approaches to health IT imple-mentation.

Table 1 includes six implementation best practices reportedin the literature that are relevant to health IT implementationand were agreed upon as a starting point during a recent work-ing group meeting held as part of the NI2012 Post-Conferenceon Implementation Science in Informatics Research. Explana-tions of the practices and their relationships to the multi-levelCFIR framework are provided in Table 1 as well.

Two health IT implementation case studies are firstdescribed and are then evaluated relative to the health ITimplementation best practices identified in Table 1. The thirdcolumn in Table 1 reflects the multi-level constructs of themodified CFIR Framework suggested by Chadoir et al. that arecausal or predictive in regards to implementation outcomes.The case studies were selected because both focus on imple-mentation of a complex health IT intervention in health care


settings. These case studies are used to highlight the multi-ple challenges faced in health IT implementation and howthese challenges can be addressed using a beginning set ofliterature-supported implementation best practices.


3. Results

In the sections below, two HIT implementation case stud-ies are first described and are then evaluated relative to thebest practices identified in Table 1 as a demonstration. Inthe discussion section, barriers and facilitators to success-ful HIT implementation are described and recommendationsare made for overcoming barriers, strengthening facilitatorsand increasing the probability of implementation success. Thecase studies highlight the multiple challenges faced in HITimplementation in very diverse environments, and how thesechallenges can be addressed using a beginning set of literaturesupported and transferrable implementation best practices.

3.1. Case Study 1: the Fall TIPS intervention

3.1.1. BackgroundFall TIPS (Tailoring Interventions for Patient Safety) is aninteractive tailored patient fall risk assessment tool designedto help nurses conduct a fall risk assessment using a validand reliable instrument, the Morse Fall Scale [35]. The FallTIPS is also designed to assist in the development and com-munication of a personalized plan based on patient-specificdeterminants of risk to all members of the healthcare teamincluding patients and family [32]. Using the web-based FallTIPS application, nurses document a patient’s fall risk factorsrelative to the following areas of risk: (1) history of falls, (2)presence of a co-morbid diagnosis, (3) presence of an intra-venous catheter, (4) use of an ambulatory aid, (5) presence ofa gait disturbance, (6) confusion or unwillingness to call forhelp. Fall TIPS displays the tailored or personalized plan that ismost likely to prevent a fall [33] based on the patient’s fall riskprofile. Fall TIPS effectiveness in reducing physiological fallswas demonstrated in a randomized control trial in four acutecare hospitals with over 10,000 patients [33]. Use of Fall TIPSresulted in a 22% reduction in patient falls and was particularlyeffective with patients over age 64 [33].

3.1.2. Best Practice 1 – identify multiple implementationmethods and modelsThe Fall TIPS project involved both development and imple-mentation of a health IT application. This required the useof multiple methods to engage stakeholders (professionaland paraprofessional providers, patients and family) in thedevelopment and implementation processes. Two frameworksguided the study; participatory design [31] and the Institutefor Health Care Improvement’s Framework for Spread (FFS)[30] Together, these frameworks helped to overcome the chal-lenges related to involving end users in iterative developmentof the intervention, engaging stakeholders in identifying waysto overcoming the inertia of previous practice, and to integratea new innovation into existing workflows [32].

The FFS is based on Roger’s Diffusion of Innovations theory[34] and supports sustained spread of messages to stakehol-ders needed to support widespread adoption and use of a new


innovation. In our earlier work, we found that the FSS waseasily integrated into existing quality management programsand was straightforward to replicate across sites [32]. Also, theFFS provided a structure that facilitated Fall TIPS adoption on




i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 4 ) xxx–xxx 5

Table 1 – Health IT implementation best practices and linkage to multilevel framework.

Identifiedbest-practice

Explanation Relative aspects of CFIRframework (Fig. 1)

1. Identify multipleimplementationmethods and models.

Methods and models are not perfect, that there is no “one sizefits all” approach to implementation. A standardizedapplication of a single implementation model is not validbecause structural, organizational, provider, patient andinnovation-level factors predict implementation outcomes[16]. Complex studies in complex settings often require theuse of multiple methods and models to supportimplementation. Methods may require adaptationmid-intervention.

StructuralOrganizationalProviderPatientInnovation(All levels can have predictive or causalimpact on implementation outcomes)

2. Collect data aboutvariation.

Variation across settings can deeply affect results – therefore itis important to not only be aware of this – but to collect dataabout variation. As noted above, multi-level factors predictimplementation outcomes. Collecting data on these factorscan facilitate understanding variation. In addition, Leykumet al. [6] assert that understanding the variation will allowthem to be “exploited in a way that will lead to maximalresults” (p. 62).


3. Identifying localchampions.

Local champions are critical for engendering adoption and usein the setting is critical. Leykum et al. [6] and others advocatefor participatory action research or cooperative inquiry.

OrganizationalProviderPatient(Levels of predictive or causal impact onimplementation outcomes)

4. Understand how themultiple levels ofcomplexinterventionsintersect and howthey relate to theintervention.

With the knowledge that complex interventions have multiplelevels and are multifaceted – implementation studies requireawareness of how the levels integrate/intersect and howthose facets relate to one another to produce the fullintervention [16].


5. Relate fidelity ofintervention tocontent and process.

In health IT implementation, the focus is upon the process ofthe implementation NOT the outcome of the intervention –the variation inherent in CAS will require considerations offidelity. It is very difficult to build a measure that yields validscores for multiple implementations or contexts, therefore abalance must be struck between efficient measurement andeffective measurement [17,39–41].

Fidelity(Focal point is upon implementationoutcome, not predictive or causalconstruct)

6. Address penetrationand sustainability aspart of the

The integration of the innovation within busy health careworkflows (penetration) and the process for operationalizingand maintaining an innovation within a service setting

impltion [

Implementation CostPenetrationSustainability

uaa

3Dtuaitrla

3Liv

implementationprocess

(sustainability) should be part of theprocess of any new health IT innova

nits and promoted sustained communication, assuring thatll stakeholders were aware and involved in adoption, spread,nd proper execution.

.1.3. Best Practice 2 – collect data about variationata were collected related to provider self-efficacy [33] related

o fall prevention and patient characteristics (collected at thenit level) that could potentially affect the outcome such asge, insurance status, and length of stay. These data providednformation about the effectiveness of the Fall TIPS interven-ion on younger versus older patients [33]. No site-specific dataelated to staffing, workload, or patient safety culture were col-ected at each site and patient characteristics were measuredt the unit level.


.1.4. Best Practice 3 – identifying local championsocal champions were identified and involved in development,mplementation, adoption, and spread of the Fall TIPS inter-ention [32]. Unit-based staff nurse champions assisted with

ementation16].

(Focal point is upon implementationoutcome, not predictive or causalconstruct)

pilot testing and with communicating and demonstratingthe advantages of using Fall TIPS in the context of typi-cal patient care workflows. Therefore, knowledge of betterideas, practices and associated benefits of the interventionwere communicated by peers rather than by research staff.Moreover, “just-in-time” in-service education provided byunit-based champions demonstrated advantages of Fall TIPSby peers who understand patient care and workflow chal-lenges.

3.1.5. Best Practice 4 – understand how the multiple levelsof complex interventions intersect and how they relate to theinterventionThe Fall TIPS intervention is complex and includes compo-nents to facilitate the three steps of the fall prevention process:


(1) fall risk assessment, (2) creating a tailored fall preventionplan, and (3) executing the plan. Each component of the inter-vention is important, but unless all three components areconsistently completed, the intervention will not prevent a




c a l i
patient from falling. Focus group interviews and observationalstudies were completed to inform the process for integrat-ing the Fall TIPS intervention into existing workflows. Theworkflows were then pilot tested and iteratively refined withend-users before the intervention was implemented. Whiledata were collected on adherence with the Fall TIPS interven-tion on the unit level (denominator was the number of patientson the unit and the numerator was the number of patientswho had the Fall TIPS toolkit components in place), no datawere collected at the individual patient level.

3.1.6. Best Practice 5 – relate fidelity of intervention tocontent and processFidelity of the intervention (e.g., adherence with the proto-col) and the use of Fall TIPS components were tracked viacomputer log files and unannounced monthly visits. Data onwhether each patient’s fall risk assessment was completed onadmission and the presence of tailored Fall TIPS informationat the bedside were collected to track adherence with the pro-tocol. A Fall TIPS Message of the Week was sent to end usersto let them know the adherence rate for their unit and to con-vey a useful tip that were communicated by end users to thestudy team. Adherence with the Fall TIPS intervention over the6-month intervention period was >90%. Qualitative methods(focus group and individual interviews) were used to evaluatesatisfaction and recommendations for improvement at eachsite. The qualitative data were used to support enhancementsto the Fall TIPS software.

3.1.7. Best Practice 6 – address penetration andsustainability as part of the implementation processBased on the evidence for Fall TIPS efficacy, the four hospitalsthat participated in the study requested to use Fall TIPS in rou-tine patient care. While use of Fall TIPS was supported by thenursing and medical leadership, the complexity of differentelectronic record systems at each hospital, created a barrier torapid deployment. The cost of operationalizing Fall TIPS fourdifferent hospitals with four different electronic systems thatwere being phased out was not seen as a viable option. A deci-sion was made to integrate the Fall TIPS application into thecommon vendor system that is scheduled to be implementedacross the enterprise over the next four years. Since the studyresults were obtained under the controlled conditions of a RCTand over a period of six months, long term sustainability inroutine care is unknown.

3.2. Case Study 2

3.2.1. BackgroundAdoption of electronic health records (EHR) in the Ngaanyat-jarra lands located in remote Western Australia highlights thecomplexity of distance and a health system controlled by anIndigenous population. The Aboriginal Health Service (AHS)in the Ngaanyatjarra lands is controlled by tribal elders andreceives state and federal government funding from variousagencies. Approximately 1500 patients receive care from the


AHS but this varies as many of the indigenous populationsthat it serves are mobile and frequently cross land boundaries.AHS employs 65 staff members and has 7 clinics across thelands. The geographic location is in the western desert area of


outback Western Australia, approximately 1000 km from AliceSprings (Central Australia) which is the main administrationcenter. It is nearly 1500 km from Perth (Western Australia cap-ital city) and is one of the most remote locations in Australia[36].

The Aboriginal Health Service delivers primary and pre-ventive care to communities which are dispersed across theseremote areas. There are no hospital or health providers basedin the central outback, and the majority of the communitiesare at least 1000 km from any regional center. The primarymethod of record keeping was paper based with records beingkept in the home communities. With a mobile indigenous pop-ulation, keeping up to date and readily accessible records wasnearly impossible. The move to electronic health records wasrelated to increasing government reporting requirements andregulations to secure future funding of the health service. Inaddition, access to digital radiology and pathology results wereon the increase, making accessibility a reality [36].

The implementation of EHRs in the remote areas of West-ern Australia was undertaken by the AHS Chief InformationOfficer (CIO), Chief Executive Officer (CEO), Chief Informa-tion Technology officer (CIT) as well as clinician input whereappropriate. The implementation was immediate rather thanphased in over time. The immediate cut-over from paperto electronic record keeping resulted in some issues withbuy-in from more traditional staff members. The CIO andCIT worked tirelessly to encourage and support clinical staffwith the intent of achieving a practical, but not necessar-ily perfect, workable system. Their understanding of theremote environment and diverse employee types ensuredthat the implementation process was tailored to the user-base. The case study reported that the implementationprocess experienced in the remote areas appeared to sur-pass the experiences reported in metropolitan areas. Whenone considers that the remote areas had much more com-plex environments complicated by distance and remoteness,extreme weather, transient populations, a lack of regular andvested employees in the health services and less than idealtechnology, it suggests that the process of implementationmay have been a defining success factor [36].

3.2.2. Best Practice 1 – identify multiple implementationmethods and modelsThis case study of the implementation of an electronichealth record (EHR) in this remote area of Western Australiabegan with an identification of the drivers, facilitators andbarriers to adoption of an EHR. Waldman’s Framework ofOrganisational Innovation Adoption [39] was used to eval-uate the outcome of the implementation and to guide theimplementation process. Barriers, facilitators, and driverswere identified as the impact of the multi-tiered healthsystem in Australia; “widespread uncertainty surroundingimplementation of EHR’s from political, policy, administra-tive, clinical and health consumer perspectives; inconsistentapproaches from government, public and private healthproviders primary and tertiary healthcare systems” [39]; the


pressures caused by the geography, the culture, and poverty.Additionally, overlap between these and other systemicissues were identified, including the diversity of stake-holders, cost of implementation, and privacy laws which




a l i n

pp

3PsfiAitiim

3Ttfmiorto(

3oiTpnwcut(aefm

3cThr

•

•••

3sTb


revented some aspects of information exchange and com-licated security [39].

.2.3. Best Practice 2 – collect data about variationost-implementation interviews were conducted with [18]taff members from the Ngaanyatjarra lands as well as staffrom the main administrative center at Alice Springs. Thesenterviews were transcribed and analyzed for major themes.dditionally, other variables from the literature that have been

dentified previously with adoption of innovation were cap-ured and assessed in an attempt to more holistically considernfluential factors. The researchers were reasonably thoroughn collecting a variety of data and observations from the imple-

entation process.

.2.4. Best Practice 3 – identifying local championshe CEO, the CIO, and the Chief IT Officer led the implementa-

ion process, with each having a different but complementaryocus. The CEO had the vision of integration of services for

obile and vulnerable populations, the CIO was the leadern relations with the staff, and the Chief IT Officer was theperational guru who kept equipment and networks up andunning in very unforgiving environments. There was no men-ion made of local champions at the individual institutionr ward levels; all local champions were from the “C-Suite”

executive level).

.2.5. Best Practice 4 – understand how the multiple levelsf complex interventions intersect and how they relate to thenterventionhe implementers were aware, from inception, of the com-lexities that would be faced in an implementation of thisature. The multiple levels and the interactions between themere considered – in example, the CEO aimed for a practi-

al system that did not require perfection, there was a deepnderstanding of the local culture and the continual migra-ion, the workforce and the environment were accommodatedlower technological competencies and connectivity barriers),nd the external forces from the government were acknowl-dged. Each of these factors (and others) were understood,rom inception, to have bearing on the outcome of the imple-

entation.

.2.6. Best Practice 5 – relate fidelity of intervention toontent and processhis case study did not directly address fidelity measures,owever there were some examples that could be said toeflect some of these principles, for example:

purposeful sampling and the selection of multidisciplinaryparticipants who had direct knowledge and use of the sys-tem that was implemented

inclusion of implementation best practices clear outcomes required of the implementation understanding of clinical requirements.


.2.7. Best Practice 6 – address penetration andustainability as part of the implementation processhe implementation process described in the case studyegan in 2004 and was extended across a large and geographic

f o r m a t i c s x x x ( 2 0 1 4 ) xxx–xxx 7

area of extreme conditions. Sustainability was not directlyaddressed in the case study, but the references to necessity ofthe effort to ensure continual governmental funding of healthservices and a deep sense of respect and compassion forthe native peoples were major drivers. Additionally, the com-mitment to a pragmatic and patient-centered approach wasrepeatedly broadcast by the local champions, and evidencedby actions undertaken during the process.

4. Discussion

The two case studies presented originate from two markedlydifferent situations. This illustrates the challenges high-lighted in the literature regarding the use of standardizedimplementation processes. “One size fits all” in health ITimplementation is a fallacy particularly when global diversityis added into the mix. However, several frameworks have beenpresented that can be used as a scaffolding to begin to assessbest practices, their distinct dimensions, and their applica-bility for use. In this fashion, a knowledge base for healthIT implementation techniques may emerge. In this regard,we use the 6 previously identified best practices (Table 1) asa schema for discussing the processes between these twomarkedly diverse implementations and we used the CFIR [16]to assure consideration of the multi-level constructs from eachsituation.

4.1. Identify multiple implementation methods andmodels

As discussed earlier in this paper, clinical environments arecomplex and variation is the norm requiring creative and crit-ical thinking and acceptance that a single model or methodmay not be sufficient for implementation studies. Adapta-tion is crucial in development, implementation and evaluationof implementation processes in CAS such as these [7]. Thetwo case studies presented used different implementationmodels. However, in both cases, the implementation mod-els supported a systematic approach to implementation thataddressed barriers and determined the data needed for eval-uation.

Two models guided the first case; participatory design [31]and the Institute for Health Care Improvement’s Frameworkfor Spread (FFS) [30]. The participatory design approach usedin Case Study 1 engaged end users (patients and providers)in iterative development of the Fall TIPS toolkit innovationand in pilot testing of the toolkit components to ensure thatthey fit within the workflow. In addition, the three phasesof the Framework for Spread: (1) planning and set-up, (2)spread within the target population and (3) continuous mon-itoring and feedback of the spread process; provided a formalmechanism to directly address causal factors from the mul-tiple organizational levels such as aligning organizationalcommunication and support for the innovation from the orga-


nizational leadership to the bedside. Identifying metrics tomeasure adoption as part of this process ensured that eval-uation occurred as an ongoing process and adaptations weremade as needed to support adoption.




c a l i
In Case Study 2, the use of Walden’s framework [39] enableda focused examination of the systemic barriers to adoptionof an EHR. This framework also suggests the use of princi-ples that can be applied to complex EHR implementations toimprove the potential for successful adoption [37]. The authorsof the case study used the framework to identify the specificfeatures that were present during the implementation andto elicit from the subjects how certain issues were overcomeduring the implementation process. Using the frameworkwas perceived as beneficial, since it allowed various dimen-sions to be assessed and measured. While the dimensionalityof the Walden Framework allowed a relatively comprehen-sive assessment, the researchers also incorporated knowledgeand findings from prior research regarding aboriginal health,behavior patterns, and existing secondary data from govern-ment sources. The incorporation of multiple methods of datafrom a variety of sources supports the best practice of identi-fying multiple implementation methods and models.

4.2. Collect data about variation

Because variation in how a health IT innovation is imple-mented and differences in the causal factors that exist ateach site can affect results, data on contextual variables areneeded. In the first case study, contextual data were collectedrelated to use of the system at the provider level but data oncertain demographics at the patient level was not collected.This was a limitation because eventually Fall TIPS was foundto be most effective with patients over age 64. Additionallybecause adherence data was collected at the unit level andnot at the individual level, it is not known whether adherencewas simply better with older patients or whether the toolkitintervention is simply more effective with this demographic.Finally, no site-specific data related to staffing, workload, orpatient safety culture was collected at each site. These datacould have potentially provided insight into the relative effec-tiveness of the intervention at each hospital [23].

In the second case study, the collection of a variety of dataresulted in the gains of valuable information regarding theimplementation of an EHRS in remote low-resource commu-nities. The structure of the EHR and then its implementationand use created some issues around cultural and lifestyleaspects for the indigenous communities. Aboriginals have anintuitive, personal and flexible concept of time, and are oftenunconcerned with schedules and order. Family structures andthe use of surnames in the Aboriginal culture are not com-patible with standard methods of patient identification in anEHRS, and deep linkages between cultural beliefs and symp-toms conflicted with structured and standardized HIT. Otheridentified issues related to remoteness of the region, lack oftechnical expertise of staff, staff turnover, lack of ability (orneed) to interact with other health information systems out-side the lands, and issues related to technical support. Theoverall point here is that the researchers cast a wide net, anddid a reasonable job of collecting data about variations thatimpacted implementation in this unique setting. Reiterating


a point made at the beginning of this paper, these findings maynot be generalizable across other distinct racial and ethnicgroups (or even other Australian Aboriginal communities) butthe concepts themselves may be transferrable. These lessons


are an important step in developing an implementation pro-cess knowledge base.

4.3. Identifying local champions

The importance of identifying and engaging local champi-ons to assist with implementation is, and has always been,a best practice in HIT implementation. In Case Study 1 thelocal champions were unit based, reflecting the importantinvolvement of those who were most impacted by the imple-mentation. These local champions participated in an iterativedesign process, communicated benefits and feasibility ofusing Fall TIPS in the context of busy patient care workflowsto their peers.

The second case study was a top-down process, wherethe local champions were high level decision makers (the so-called “C-Suite” executives) The CEO and CIO from the healthservice were the champions for this case study as they drovethe implementation and adoption of the system. They hadthe knowledge, skills and understanding of the complexitiesof the healthcare system, the remote environment, employeeand employer requirements and the passion for innovationand better health outcomes for the local communities. Fromthe case study it is evident that their passion, motivation andprofessionalism was a major reason for the successful imple-mentation and adoption of the EHR system. The balancing offinishing up with a workable system – as opposed to a per-fect system – was also very valid and pragmatic. Finally, byfocusing on measures of success and maintaining the patient-centric mission engendered buy-in from the initially hesitanttraditionalists. It is interesting to note however that there wasno mention in the report of end-user involvement. This may,in of itself, be an important piece of knowledge, relevant toHIT implementation in low resource areas. HIT deploymentmay be dictated by Ministries of Health or by funding agencies,reporting structures are often rigid and hierarchical, long termsustainability and commitment can be uncertain, and politi-cal whims may dictate implementation approaches. It maybe that a top-down implementation of EHRS in low-resourceareas is a best practice in of itself. Further study is warranted.

Ultimately, these two case studies were successful, due inpart to strong local champions. Does the identification of abest practice require dictating who those local champions are?These two case studies would seem to imply that the localchampions can be top-down or bottom-up, or somewhere inthe middle. Much depends on the situation, context, and cul-ture, particularly in the case of country variation. Complexityand variation of the situation influences who the local cham-pions are – again illustrating the necessity of flexibility andadaptation in EHRS implementation interventions.

4.4. Understand how the multiple levels of complexinterventions intersect and how they relate to theintervention

As discussed earlier, health IT innovations are frequently com-


prised of multiple levels or components. This aspect heavilyinfluenced Case Study 1 and presented a challenge for fullyunderstanding how the different components make up a“dose” of the fall prevention intervention. One way to address




a l i n

ttwduihfpta

WmTaagatsaitsbmEpecpt

4p

Tmatfiiwanfi

hpccctp

utit


his challenge was to identify a way to measure “dose” withinhe study. This was a major limitation in the Fall TIPS study,here adherence data were collected at the unit level, but noata related to dose (e.g., the extent to which each individ-al patient received the three components of the Fall TIPS

ntervention) were collected at the patient level. There is aigh probability that the influence of different units and dif-

erent institutions also played a role in the implementationrocess, however the lack of data delineating the specific loca-ion precluded a deeper understanding of potential barriersnd facilitators to implementation outcomes.

In Case Study 2, the systemic issues identified withinalden’s framework [38] revealed the complexity in imple-enting a large information technology projects such as this.

he additional challenges of geography, dealing with vulner-ble and mobile populations, and the complexities of culturedded to the difficulty. The results of the study however sug-ested that similar implementations in complex metropolitanreas (where interfacing disparate systems massively increasehe complexity) were more difficult than those in the regionerved by the Aboriginal Health Service. While this may ben artifact of isolation, the introspection of the researcherss an important one. Regarding the region as a discrete sys-em (i.e., totally isolated), enabled many of the complex issuesuch as multi-system or multi-vendor interoperability to beypassed [36] which in turn eased certain aspects of the imple-entation. Eventually these issues will require attention as

HRS roll-out across Australia, but the lessons learned in thisarticular implementation process generated valuable knowl-dge for implementations of EHRS in low-resource and remoteommunities. In addition, the findings related to working withatients from markedly non-Western cultures provide impor-ant knowledge for EHRS implementation personnel.

.5. Relate fidelity of intervention to content androcess

o encourage adoption and sustained use of implementationodels in EHRS interventions, the focus must shift from rigid

dherence of a “one size fits all model” to an adaptive approachhat takes into account situational context while maintainingdelity of the intervention. Strategies that consider how EHRS

nvention processes can be adapted to meet unique situationshile still maintaining enough rigor to insure that the findings

re transferable are desperately needed. This dimension wasot a focal point for either case study, however, evidence ofdelity measurements are present in both.

In the first case study, standard operating procedureselped to maintain fidelity and adherence to the studyrotocols. Adaptation to the implementation process wasontrolled in a way that enabled flexibility to the core pro-ess without jeopardizing fidelity. Multiple methods includingomputer log files and site visits were used to measure fidelityo the intervention. In addition, interviews were conducted torovide context into the quantitative findings.

The second case study failed to formally address meas-


res of fidelity, although as noted earlier, they did reporthe study as “successful” and they did conduct a series ofnterviews, secondary data analysis, and review of litera-ure related to aboriginal health care. One would assume


that they maintained fidelity, but without mention of it,this aspect is hard to judge. It would have strengthenedthe study had the researchers used an evaluation tool thatincluded the underlying fidelity principles of purpose andtheory, adherence, data collection methodology, quality ofdelivery, participant responsiveness and program differenti-ation [7,39] to ensure readers of fidelity. While the points aremade in the literature that when high fidelity is not achievedthe impacts of an implementation cannot be identified, recti-fied or improved [39,41] adaptive mixed-model approaches arerequired to gain the level of understanding required in EHRSinterventions.

4.6. Address penetration and sustainability as part ofthe implementation process

Penetration and sustainability were important themes in bothof the case studies. For the first case study, the adherencelevel with the intervention of >90% during the clinical trialsuggested good penetration. However, sustainability was notachieved due to the cost of implementing the intervention inelectronic systems that were being phased out. This is notto imply that the lessons learned during the implementa-tion were all for naught. However, the Fall TIPS effort wastied to a distinct system that was eventually replaced in theorganization.

The second case exemplifies the importance of executivelevel buy-in to the project, and illustrates how a project of thistype, driven by governmental requirements coupled with per-sonal commitment can impact penetration and sustainability.The outcome was positive, the penetration was extensive– spreading out across the Ngaanyatjarra lands of WesternAustralia, and the system is still functioning, a testimony to itssustainability. Part of the success however is related to the iso-lation of the outback region and the relatively slow uptake ofwidespread and integrated EHRS across Australia. This has notpressured the Western Region to become interoperable withthe remainder of the nation. Eventually the isolation that wassuch a benefit for implementing EHRS in Western Australiawill begin to take its toll. The effort that it will take to linkinto other health information systems outside the region willbe considerable, and the adaptations to a more Westernizedmodel of care will impose new challenges for the AboriginalHealth Service.

5. Summation

The presentation, analysis and discussion illustrated that thenascent six implementation best practices and the CFIR wereacceptable structures for assessing certain aspects of the casestudies. There is much work left to be done, not only in gaininga deeper understanding of implementation processes, but alsoin developing a scaffold or method for beginning to analyze


EHRS implementations as we have begun. Interesting findingsemerged from both, and using the scaffold (Table 1) assistedus in deconstructing, examining, organizing and representing2 very different implementation processes.




10 i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i

Summary pointsWhat is already known?

• There is more than evidence needed for successfuladoption, implementation and sustainability of healthIT.

• There is a pressing need for better understanding ofwhat works, what does not, and in what context.

• Mixed models and mixed methods may work best forimplementation interventions.

What does this study add?

• A beginning set of literature supported and trans-ferrable implementation best practices are put forwardfor use and evaluation.

• CFIR is suggested as a good framework for implemen-

r

l

tation research.

6. Conclusion

Adoption of electronic systems and other health IT innova-tions into complex healthcare environments has been slowand problematic. Recent work in implementation science sug-gests that both the technology and the implementation pro-cess must be addressed when implementing health IT inno-vations. This paper examined health IT implementation pro-cesses and identified a beginning set of implementation bestpractices, which if used consistently, could begin to addressgaps in the health IT implementation body of knowledge.

Author contributions

Patricia Abbott is the lead author who organized submission,managed the project, wrote significant portions of the paper,participated in editing, participated in development of paperframework/grid and identification of six best practices, andsubmitted the paper. Jo Foster, Patricia Dykes and HeimarMarin contributed in writing/summarizing one of the two casestudies, editing, participating in identification of six best prac-tices, participating in development of paper framework/gridand conducting portions of the literature review.

Conflict of interest

There are no conflicts of interest noted from any of the authorson this paper.

e f e r e n c e s

[1] A. Kitson, G. Harvey, B. McCormack, Enabling theimplementation of evidence based practice: a conceptualframework, Quality in Health Care 7 (1998) 149–158.


[2] Y. Han, J. Carcillo, S. Venkataraman, R. Clark, R. Watson, T.Nguyen, H. Bayir, R. Orr, Unexpected mortality afterimplementation of a commercially sold computerizedphysician order entry system, Pediatrics 116 (2005) 1506.


[3] T. Yackel, P. Embi, Unintended errors with EHR-based resultmanagement: a case series, Journal of the American MedicalInformatics Association 2010 (17) (2009) 104–107.

[4] R. Koppel, J.P. Metlay, A. Cohen, B. Abaluck, A.R. Localio, S.E.Kimmel, et al., Role of computerized physician order entrysystems in facilitating medication errors, Journal of theAmerican Medical Association 293 (10) (2005) 1197–1203.

[5] A. Chapman, C. Lehmann, P. Donohue, S. Aucott,Implementation of computerized provider order entry in aneonatal intensive care unit: impact on admission workflowOriginal Research Article, International Journal of MedicalInformatics 81 (5) (2012) 291–295.

[6] A. Kellerman, S. Jones, Analysis and commentary: what itwill take to achieve the as-yet-unfulfilled promises of HealthInformation Technology? Health Affairs 2013 (January (32))(2013) 63–68.

[7] L. Leykum, J. Pugh, H. Lanham, J. Harmon, R. McDaniel,Implementation research design: integrating participatoryaction research into randomized clinical trials,Implementation Science 4 (2009) 69.

[8] K. Unertl, L. Novak, C. Gadd, N. Lorenzi, The science behindhealth information technology implementation:understanding failures and building on success, AMIAProceedings (2012).

[9] F. Mair, C. May, C. O’Donnell, T. Finch, F. Sullivan, E. Murray,Factors that promote or inhibit the implementation ofeHealth Systems: an explanatory systematic review, Bulletinof the World Health Organization 90 (5) (2013) (accessed21.04.13)http://www.who.int/.bulletin/volumes/90/5/11-099424/en/

[10] J.E. Richardson, E.L. Abramson, E.R. Pfoh, R. Kaushal, HITECInvestigators, Bridging informatics and implementationscience: evaluating a framework to assess electronic healthrecord implementations in community settings, AMIAAnnual Symposium Proceedings 2012 (2012)770–778.

[11] K. Keshavjee, C. Kuziemsky, K. Vassanji, A. Ghany, Acomplex adaptive systems perspective of healthinformation technology implementation, in: K.L. Courtney,et al. (Eds.), Enabling Health and Healthcare Through ICT,IOS Press, Amsterdam, Netherlands, 2013.

[12] L.A. Lipsitz, Understanding health care as a complexsystem: the foundation for unintended consequences,Journal of the American Medical Association 308 (3) (2012)243–244, http://dx.doi.org/10.1001/jama.2012.7551.

[13] National Information Center on Health Services Researchand Health Care Technology (NICHSR),http://www.nlm.nih.gov/hsrinfo/implementation science.htm

[14] L. Damschroder, D. Aron, R. Keith,S. Kirsch, J. Alexander, J. Lowery, Fostering implementationf health service research finding into practice:a consolidate framework for advancing implementationscience, Implementation Science 4 (50) (2009).

[15] E. Ferlie, S. Shortel, Improving the quality of healthcare inthe United Kingdom and the United States: a framework forchange, Milbank Q 79 (28) (2001) 281–315.

[16] S. Chaudoir, A. Dugan, C. Barr, Measuring factors affectingimplementation of health innovations: a systematic reviewof structural, organizational, provider, patient, andinnovation level measures, Implementation Science 8 (2013)22.

[17] P. Nielsen, K. Roback, A. Boström, P. Ellström, Creatures ofhabit: accounting for the role of habit in implementationresearch on clinical behavior change, ImplementationScience (7) (2012) 53.


[18] S.K. Schoenwald, A.F. Garland, J.E. Chapman, S.L. Frazier, A.J.Sheidow, M.A. Southam-Gerow, Toward the effective andefficient measurement of implementation fidelity,



http://refhub.elsevier.com/S1386-5056(13)00227-X/sbref0005































































































































































































































































































































































http://www.who.int/.bulletin/volumes/90/5/11-099424/en/




















































































































dx.doi.org/10.1001/jama.2012.7551

http://www.nlm.nih.gov/hsrinfo/implementation_science.html

















































































































































































































































l i n

[41] C. Mowbray, M. Holter, G. Teague, D. Bybee, Fidelity criteria:development, measurement and validation, American

i n t e r n a t i o n a l j o u r n a l o f m e d i c a

Administration and Policy in Mental Health 38 (January (1))(2011) 32–43.

[19] C. Phibbs, A. Milstein, S. Delbanco, D. Bates, Published lettersto the editor: no proven link between CPOE and mortality,Pediatrics 116 (December (6)) (2005) 1512–1513.

[20] C.R. May, F.S. Mair, T. Finch, A. Macfarlane, C. Dowrick, S.Treweek, et al., Development of a theory of implementationand integration: normalization process theory,Implementation Science 4 (2009) 29,http://dx.doi.org/10.1186/1748-5908-4-29, PMID: 19460163.

[21] L. McQueen, B.S. Mittman, J.G. Demakis, Overview of theVeterans Health Administration (VHA) quality enhancementresearch initiative (Queri), Journal of the American MedicalInformatics Association 11 (5) (2004) 339–343.

[22] R.E. Glasgow, L.M. Klesges, D.A. Dzewaltowski, P.A.Estabrooks, T.M. Vogt, Evaluating the impact of healthpromotion programs: using the RE-AIM framework to formsummary measures for decision making involving complexissues, Health Education Research 21 (5) (2006) 688–694.

[23] S. Bakken, C. Ruland, Translating clinical informaticsinterventions into routine clinical care: how can the RE-AIMframework help? Journal of the American MedicalInformatics Association 16 (November–December (6)) (2009)889–897.

[24] G. Aarons, A. Green, L. Palinkas, S. Self-Brown, D. Whitaker,J. Lutzker, J. Silovsky, B. Hecht, M. Chaffin, Dynamicadaptation process to implement an evidence-based childmaltreatment intervention, Implementation Science 7(2012) 32.

[25] G.A. Aarons, M. Hurlburt, S.M. Horwitz, Advancing aconceptual model of evidence-based practiceimplementation in child welfare, Administration and Policyin Mental Health 38 (2011) 4–23.

[26] L.L. Novak, R.J. Holden, S.H. Anders, J.Y. Hong, B.T. Karsh,Using a sociotechnical framework to understandadaptations in health IT implementation, InternationalJournal of Medical Informatics 82 (12) (2013, Dec) e331–e344.

[27] M.I. Harrison, R. Koppel, S. Bar-Lev, Unintendedconsequences of information technologies in health care aninteractive sociotechnical analysis, Journal of the AmericanMedical Informatics Association 14 (2007) 542–549.

[28] M. Berg, J. Aarts, J. van der Lei, ICT in health care:sociotechnical approaches, Methods of Information inMedicine 42 (4) (2003) 297–301.

[29] C.M. Ruland, R.M. Maffei, E. Børøsund, A. Krahn, T.Andersen, G.H. Grimsbø, Evaluation of different features of


an eHealth application for personalized illness managementsupport: cancer patients’ use and appraisal of usefulness,International Journal of Medical Informatics 82 (7) (2013, Jul)593–603.


[30] M.R. Massoud, G.A. Nielsen, K. Nolan, M.W. Schall, C. Sevin,A Framework for Spread: From Local Improvements toSystem-Wide Change IHI Innovation Series White Paper,Institute for Healthcare Improvement, Cambridge, MA, 2006www.IHI.org

[31] C. Ruland, J. Starren, T. Vatne, Participatory design withchildren in the development of a support system forpatient-centered care in pediatric oncology, Journal ofBiomedical Informatics 41 (4) (2008) 624–635.

[32] P.C. Dykes, D.L. Carroll, A.C. Hurley, A. Benoit, B. Middleton,Why do patients in acute care hospitals fall? Can falls beprevented? Journal of Nursing Administration 39 (June (6))(2009) 299–304.

[33] P.C. Dykes, D.L. Carroll, A. Hurley, S. Lipsitz, A. Benoit, F.Chang, S. Meltzer, R. Tsurikova, L. Zuyov, B. Middleton, Fallprevention in acute care hospitals: a randomized trial,Journal of the American Medical Association 304 (November(17)) (2010) 1912–1918.

[34] E.M. Rogers, Lessons for guidelines from the diffusion ofinnovations, Joint Commission Journal on QualityImprovement 21 (July (7)) (1995) 324–328.

[35] J. Morse, Preventing Patient Falls, Sage Publishing Co., CA,1997.

[36] H. Cripps, C. Standing, The implementation of electronichealth records: a case study of bush computing theNgaanyatjarra Lands, International Journal of MedicalInformatics 80 (12) (2011) 841–848.

[37] S. Standing, C. Standing, Mobile Technology and healthcare:the adoption issues and systemic problems, InternationalJournal of Electronic Healthcare 4 (3–4) (2008)221–235.

[38] J. Waldman, Thinking systems need systems thinking,Systems Research and Behavioral Science 24 (3) (2007)271–284, http://dx.doi.org/10.1002/sres.828.

[39] C. Carroll, M. Patterson, Wood E., S. Booth, R. Balain, Aconceptual framework for implementation fidelity,Implementation Science 2 (40) (2007),http://dx.doi.org/10.1186/1748-5908-2-40http://www.implementationscience.com/content/2/1/40

[40] J. Century, M. Rudnick, F. Cassie, A framework for measuringfidelity of implementation: a foundation for sharedlanguage and accumulation of knowledge, American Journalof Evaluation 31 (2) (2010) 199–218,http://dx.doi.org/10.1177/1098214010366173.


Journal of Evaluation 24 (3) (2003) 315–340,http://dx.doi.org/10.1177/109821400302400303.
























































































































dx.doi.org/10.1186/1748-5908-4-29























































































































































































































































































































































































































































http://www.ihi.org/








































































































































































































































































































dx.doi.org/10.1002/sres.828

dx.doi.org/10.1186/1748-5908-2-40

http://www.implementationscience.com/content/2/1/40

dx.doi.org/10.1177/1098214010366173

dx.doi.org/10.1177/109821400302400303

Complexity and the science of implementation in health IT-Knowledge gaps and future visions

Documents

Transcript of Complexity and the science of implementation in health IT-Knowledge gaps and future visions