43 (2007) 1305–1320www.elsevier.com/locate/dss

Decision Support Systems

Doing more with more information: Changing healthcare planningwith OLAP tools

Monica Chiarini Tremblay a,⁎,1, Robert Fuller a,2, Donald Berndt a, James Studnicki b

a Information Systems and Decision Sciences Department, College of Business Administration, 4202 Fowler Ave., CIS1040,University of South Florida, Tampa, FL 33620, USA

b College of Public Health, University of South Florida, Tampa, FL 33620, USA

Available online 23 March 2006

Abstract

On-line analytical processing (OLAP) is an example of a new breed of tools for decision support that give decision makers theflexibility to customize the selection, aggregation, and presentation of data. To understand the impact of this type of tool, we studyan implementation of an OLAP interface on the CATCH data warehouse used by knowledge workers at a regional health planningagency in the State of Florida. The results of a qualitative field study show that after the OLAP implementation, these workersmade use of the additional capabilities of OLAP (e.g., aggregation levels and intuitive data manipulation), thereby leveraging theirindividual abilities to enhance and expand on the tasks they performed for their community. Consequently, they were able toperform in more of a consultative role to their clients, and improved their reputation in the community they serve. This researchadds a new dimension to prior research in data warehousing by focusing on the decision support capabilities of OLAP.© 2006 Elsevier B.V. All rights reserved.

Keywords: Healthcare planning; OLAP; Data warehouse; Decision support systems; Case study

1. Introduction

Previous research on decision support systems hasfocused on technologies that assist managers in makingdecisions. Many of these technologies can be considered“static” tools since the data presentation and analysis arepredetermined by the designers of the technology and

⁎ Corresponding author.E-mail address: mtrembla@coba.usf.edu (M.C. Tremblay).

1 Present address: Department of Decision Sciences and InformationSystems, Florida International University, 11200 SW 8th Street,Miami, FL 33199.2 Present address: Department of Accounting and Information

Management, The University of Tennessee, 636 Stokely ManagementCenter, Knoxville, TN 37996, United States.

0167-9236/$ - see front matter © 2006 Elsevier B.V. All rights reserved.doi:10.1016/j.dss.2006.02.008

unlikely to change after implementation. Today, organiza-tions are implementing a new breed of tool for decisionsupport. These technologies are more malleable in thatthey allow decision makers to work with vast amounts ofdata and give them the ability to modify how and whichdata are presented and, to some degree, to select themethods used to analyze data. Unlike earlier technologies,a systems analyst or programmer does not completelypredetermine the way a decision-maker can view oranalyze data [4,16]. An example of such a technology ison-line analytical processing (OLAP). OLAP tools allowknowledge workers to manipulate and perform complexanalyses over the data stored in a data warehouse.

Data warehouses contain a wealth of data, and tend tobe quite complex in their architecture. Studies on theimplementation of data warehouses [8,17,20,26] have

1306 M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

often investigated success factors for initial implementa-tions of data warehouses. However, these studies usuallydo not consider the tools that are typically used to accessand analyze the data stored in the data warehouse (such ason-line analytical processing, OLAP) – the tools used toreport on the knowledge or “business intelligence”contained in the system. These enabling technologiesallow users to mine and present data in a meaningfulformat [19] to extract benefit from data warehouses.Likewise, articles in the practitioner journals indicate thatwhile companies may be successful in implementingmulti-terabyte data warehouses, they are finding that it isdifficult to translate this investment into useful businessknowledge or positive business results [13]. Glasseyacknowledges that some of the most elegantly designeddata warehouses sit abandoned because the developers donot provide the correct end-user tool and they fail toengage and enable users. She identifies the need for toolsthat enable “user-centric” computing [10].

Unlike past research into data warehouses whichgenerally do not consider the OLAP front-end, our focusin this study is to understand how OLAP functionalityimpacts the nature of the work (knowledge work)performed by health planners at a regional healthplanning agency in the State of Florida. These agenciesare tasked with overseeing the Certificate of Need(CoN) program and developing their district health plan,as well as addressing issues relating to the un-served andunder-served populations in their community. As part oftheir community healthcare assessments and healthcareneeds planning, health planning agencies collect datafrom a variety of sources and employ various tools for dataanalysis and presentation purposes. Many of their tasksentail combining data from hardcopy reports provided bythe State of Florida, web pages, and publications, whichdo not lend themselves to easy aggregation and analysis.

To examine the context in which health planningagency employees operate, and how this new technol-

Technology

Individual

Task

Task-Technolog

Technology

Individual

Task

Task-Technolog

Fig. 1. Task-technology

ogy impacts them, we use Task-Technology Fit Theory(TTF) as a framework to identify the relevant constructsin this context. TTF suggests that information systemswill have a positive impact on performance when theirfunctionality is appropriately matched to user character-istics and task requirements [11,12] (see Fig. 1).Therefore, with a better match between a task'scharacteristics, the features of the technology, and thecharacteristics of the individuals involved, a higher levelof performance should occur.

This study is unique in three ways: 1) it examines theimplementation of an OLAP interface and underlyingdata warehousing technology within a health planningagency, the first implementation of its kind in the Stateof Florida, 2) it identifies the potential of these tools byexamining their use in a real healthcare planning setting,and 3) analyzes at an individual level of analysis theimpact of these tools. Past studies of similar implemen-tations [8,17,20,26] do not make use of an individuallevel of analysis, but rather utilize an organization level ofanalysis. This perspective allows us to investigate at theindividual level the impact of this technology on health-care planning activities. Furthermore, this research adds anew dimension to prior research in data warehousing byexploring how OLAP technologies can contribute to thesuccessful implementation of a data warehouse.

The paper continues as follows. First, we brieflyexamine prior research on data warehouses and inparticular business intelligence tools such as OLAP, andtheir potential benefits. Next, we provide an overview ofhealth planning agencies and illustrate one of the majorfunctions they perform, the Certificate of Need. We alsodescribe the healthcare data warehouse used in thisstudy, the CATCH data warehouse. We outline ourresearch methodology and report our findings based ona series of structured interviews conducted over a 1-yearperiod. We utilize quotes from these interviews andscreen shots from the OLAP interface to describe the

y FitPerformance

y FitPerformance

fit (Goodhue [11]).

1307M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

nature of the work of health planners, the tasks theyperform, both before and after the implementation ofOLAP capabilities. We conclude with a discussion ofthe results, and implications for future research in thisarea.

2. Data warehouses and OLAP

Data warehousing and on-line analytical processing(OLAP) [3] are fairly recent technological advances inthe field of information systems for decision support.Data warehouse systems were built to assist businessusers in analyzing the vast amount of data that originatefrom heterogeneous data sources, especially operationaltransaction processing systems that record the day-to-day activities of most organizations [23]. OLAPinterfaces not only integrate the functionality of earliergenerations of data warehouse information technologybut also go further and introduce spreadsheet-likemultidimensional data views and graphical presentationcapabilities [16]. OLAP interfaces provide a fairlysimple, yet extremely flexible navigation and presenta-tion environment that is capable of representing thestructure of real world data typically stored in the formof multidimensional tables or data cubes.

A data cube implemented as a relational star schemahas two kinds of entities: facts and dimensions. The setof dimensions consists of many categorical or textualattributes. For example, a customer dimension maycontain a key, name, purchase profile, credit profile,address, salary level, home ownership information,along with possibly hundreds of other characteristics.The fact tables are populated with numeric attributeslike dollars sold, units sold, costs, revenues, and anyother attributes that can be summarized using mathe-matical operators. Dimensions usually have associatedhierarchies that specify different aggregation levels [18].Interesting patterns are often highlighted by analyses atwell-chosen levels of aggregation. Dimensions alsoallow a user to ‘slice and dice’ a data cube to formcustomized data subsets for further analysis.

OLAP interface tools take advantage of the predict-able structure of data cubes, designed with dimensionsand facts, to provide a graphical query environment.Typically, these tools generate highly structured SQLqueries, often using analytic query language extensions,which organize and aggregate a subset of data from thewarehouse. OLAP users can navigate through a cube byeither “drilling-down” (focusing on a lower level ofgranularity, e.g., dollars sold on a certain day of theweek for a certain promotion) or “rolling up” (choosinga higher level of aggregation) [14]. Manipulation and

presentation of such information is performed by endusers through these multidimensional views and graph-ical displays which provide invaluable support fordecision-making [16]. Although the data cube is asimple structure, the large numbers of alternatives,including many numeric facts, many dimensions thatcan be used to slice through the data, and manyhierarchies or abstraction levels combine to form animmense universe of queries that can be explored via anOLAP interface.

3. Study context–health planning agencies

In the State of Florida, local health planning agencieswere established as a network of non-profit agencies thatconduct regional health planning and implementationactivities. The Boards of Directors of these agencies arecomposed of healthcare providers, purchasers andnongovernmental consumers. Florida's eleven healthplanning agencies (ranging in size from 1 county to 16counties) develop district health plans containing data,analyses and recommendations that indicate the health-care status and needs in the community for local healthcare facilities. Their recommendations are designed toimprove access to healthcare, reduce disparities inhealth status, assist state and local governments in thedevelopment of sound and rational healthcare policies,and advocate on behalf of the underserved. Local healthplanning agencies also study the impact of variousinitiatives on the healthcare system, provide assistanceto the public and private sectors, and create anddisseminate materials designed to increase their com-munities' understanding of healthcare issues [22].

3.1. Certificate of need

One of the important health planning agency func-tions is to provide detailed information for Certificateof Need (CoN) reviews. The Certificate of Needreview process regulates entry and expansion of avariety of health services by state-licensed hospitals,nursing homes, and other organizations. The CoNprogram in Florida, as well as in other states, wasoriginally created in response to federal healthplanning standards (see www.fdhc.state.fl.us/MCHQ/CON_FA). Furthermore, a state was required to have aCoN program in order to qualify for federal U.S.Public Health Service funds. The objective of CoNprograms is to partially regulate the services offered byhealthcare provider organizations in a specified area,making sure healthcare needs are met, but avoidingcostly duplication of services.

1308 M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

The Certificate of Need program is one of the explicitmechanisms for directly influencing the allocation ofservices in a healthcare marketplace. The factorsconsidered in a Certificate of Need review include: thehistory and current services of the applicant organization,the healthcare needs of the local population, resourcesavailable to the applicant organization, possible alterna-tive approaches, and any impacts on local healthcarecosts. Additionally, the CoN review process considersexcess capacity that can lead to over-utilization. CoNreviews can identify the existence of excess surgicalprograms that could lead to more, possibly unnecessary,surgeries. Other objectives include the containment ofhealthcare costs, providing equal access to care, andfostering continual improvements in the quality of care.There is much a stake during the CoN review process,with stakeholders on both sides of the issues scrutinizingthe work of fact-finding agencies such as the healthplanning agencies. Because of the data intensiveness,detailed analyses, and planning needed for CoN reviews,these types of activities are well-suited to the applicationof data warehousing and the analytic capabilities ofOLAP interfaces.

3.2. The CATCH data warehouse

Since 1991, the College of Public Health at theUniversity of South Florida has developed and en-hanced a methodology for performing communityhealth assessments – the Comprehensive Assessmentfor Tracking Community Health (CATCH). TheCATCH methodology identifies both a rich collectionof quantitative indicators and a framework for rankinghealthcare problems to support policy formulation[16,17]. The CATCH methodology has been appliedto more than two-dozen counties throughout Florida, aswell as selected applications in other states, with greatsuccess. For each of the hundreds of healthcare indi-cators in the CATCH methodology, the selected countyis compared to the state average and an average based onstatistically selected peer counties in the state. All indi-cators that are worse than both the state average and thepeer counties' average are subjected to further analysesand prioritization. The final report provides a rank-ordered list of healthcare indicators that represent aprofile of the communities' most serious and challeng-ing health problems. Originally, these CATCH reportswere prepared by hand, a difficult and time-consumingtask, spurring the development of an integrated, state-wide healthcare data warehouse.

A data warehouse project was initiated in 1998 toautomate much of the community assessment process,

although the interpretation of the many health statusindicators is still a matter of art. Today, the data warehouseintegrates fine-grained event data such as vital statistics(birth and death records), hospital discharge data, andfree-standing clinic data, along with several more detaileddisease registries. The infrastructure also includes majorhealthcare coding systems, such as the InternationalClassification of Disease (ICD), Common ProcedureTerminology (CPT), and Diagnostic Related Group(DRG) coding systems to support analyses at varyinglevels of detail. In addition, population data from thecensus, economic data, and even commercial marketingdata are added to the mix.

Early in 2000, the federal Health Resources andServices Administration (HRSA) awarded three consec-utive 1-year grants to a consortium of health-relatedorganizations within the health planning agency's regionto establish the Primary Care Access Network (PCAN).The PCAN established a series of coordinated activitiesin a multi-county area that were intended to provideaccess to primary care services, largely for the uninsuredor underinsured, which would enable the medicallyindigent to receive important primary treatment andpreventive services in defined “medical homes” andavoid inappropriate use of hospital emergency rooms fornon-emergency situations. One important objective ofthe PCAN was to perform baseline assessments of thehealth status in its various constituent communities.PCAN contracted with the University of South Florida(USF), Center for Health Outcomes Research, toevaluate the progress of various PCAN activities and,subsequently, to perform two specific community healthstatus assessments. Three reports, a special analysis andprojection of the number of uninsured, their healthcareutilization, and healthcare costs, were undertaken.

These three major reports proved valuable to thehealth provider community, providing important objec-tive documentation for multiple grant applications in thecommunity. However, they also stimulated the need formore specific information than could be provided in thehardcopy reports, as extensive as they were. Further-more, the health planning agency was designated toserve the role of information provider for health planningand evaluation purposes across the multi-county area.The health planning agencywould play an important rolein health assessments and various regulatory activities,such as Certificate of Need reviews.

Given these increased needs for data to serve a varietyof planning and assessment purposes, the health planningagency and the developers of the CATCH data warehousedecided that the health planners would benefit enor-mously by having direct access into the data warehouse.

1309M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

This would allow the planners to proactively respond totheir many requests for information and to better meettheir assessment and CoN review responsibilities, ratherthan relying on hardcopy periodic assessment reports. Inthis study, we observed the implementation and use of anOLAP interface on the CATCH data warehouse byagency health planners within a specific regional healthplanning agency in Florida. We follow with a descriptionof how we studied this implementation and collected andcoded the resulting data.

4. Research methodology

It has been noted by W. Whyte and K. Whyte [24]that behavior is best understood by studying it in itscontext. OLAP tools provide highly interactive andmalleable interfaces whose effectiveness is linked to thetasks for which they are used, and to the characteristicsof the individuals who use these tools. Additionally,individuals using this technology in the healthcareindustry can have significant impact on the communitywhich they serve. As a result, the context of use isimportant in understanding both how knowledge work-ers, such as health planners, can be aided by OLAP toolsand the impact of the usage of these tools on thecommunities in which they serve. Yin [27] describesusing qualitative research methods when you wish toinvestigate “a contemporary phenomenon within itsreal-life context, especially when the boundariesbetween phenomenon and context are not clearlyevident”. Therefore, we chose a field study approachas an appropriate method to examine the use of theOLAP interface and underlying data warehouse by thehealth planning agency, given the few studies that havebeen carried out in this area [1].

A single research site was chosen since this was aninitial attempt [25] to describe the impact of providing amalleable decision support tool such as OLAP toknowledge workers in the healthcare industry. Theresearch is based on a 1-year field study conductedbetween October 2003 and October 2004. During thistime period, knowledge workers at a Florida healthplanning agency were provided with access to a datawarehouse in an iterative fashion, beginning withhardcopy reports and ending with direct access to datacubes through an OLAP interface that gave them theflexibility to select and customize the data presentation.

4.1. Data collection method

Multiple data collections methods were used in thisstudy, including: interviews and direct observation, as

well as documents, which were used to corroborate andaugment evidence from our other sources [27]. Weconducted three interviews with each of the four healthcouncil workers and recorded three different trainingsessions. The interviews were semi-structured, allowingfor open-ended responses. The interviews were guidedby the constructs from TTF and consisted of a set ofquestions regarding the individual's previous workexperience and training, their current job activities, thetools they used to perform their tasks prior to using theOLAP interface, their expectations of OLAP, theirfamiliarity with OLAP-type tools (e.g., spreadsheets andpivot tables) and their preference for self-initiated datacollection and aggregation.

The number of interviews to conduct was notdetermined a priori. Since the technology was imple-mented in stages, interviews were administered prior toinitial implementation as well as after each implemen-tation stage. Most interviews were conducted at thehealth council's offices. During the interviews, partici-pants were asked to illustrate examples of projects theywere working on and their data collection and analysistechniques. Each time a data cube was implemented orexpanded, telephone interviews were conducted. Theseinterviews were open-ended discussions where theparticipants were asked whether access to the datawarehouse through the OLAP interface was aiding themin their data collection and analysis efforts. Therespondents were asked to cite particular exampleswhere their work load was significantly reduced,increased or altered, and instances where they wereable to perform tasks with the data that would not havebeen possible prior to the availability of the OLAP tool.Of particular interest were situations where they wereable to provide data to their clients that made asignificant impact on the communities they served.With the permission of the participants, these interviewswere recorded and later transcribed and coded.

The researchers were also present during the trainingsessions held by the developer. These training sessionswere also recorded and later coded. As noted above, thegoal was to gain an understanding of how well theparticipants understood the technology and to capturetheir questions and concerns about the use of the OLAPinterface capabilities in their work.

4.2. Data preparation and analysis

The interviews, training sessions and documentswere coded using template analysis. This technique wasselected because of its flexibility. Unlike a groundedtheory approach [21], template analysis normally starts

1310 M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

with at least a few predefined codes which help guideanalysis. The first step in template analysis is creating aninitial template by exploring the interview scripts,academic literature, the researcher's own personalexperience, anecdotal and informal evidence, and otherexploratory research [15]. The initial template wasguided by the literature on TTF which identifies anddefines our constructs of interest in this study: the tasks,technologies, individuals characteristics, level of fit, andsubsequent outcomes [12]. Once the template wascompleted, the researchers listened to the taped inter-views and worked through the provided documents,identifying sections that were relevant and annotating theappropriate codes from the initial template. The initialtemplate was modified as needed. Several iterations ofthis step led to a final template. As recommended byKing [15], coding was complete when all researchersagreed on all higher ordered codes and most lower ordercodes.

The final task is to present the researcher's interpre-tation of the data. King [15] outlines three commonapproaches. The approach taken was to create an accountstructured around the main themes identified, drawingillustrative examples from each transcripts as required.As recommended by King [15], direct quotes fromparticipants are included: short quotes to aid in thespecific points of interpretation and longer passages ofquotation to give a flavor of the original tasks.

5. Supporting healthcare planning

In this section we utilize the constructs from TTF todescribe the work context prior to implementation andthe level of fit after implementation. We begin by

Task Characteristics

Knowledge Acquisition Knowledge Packaging

Fit

Individual Characteristics

Creativity Experience Skills Autonomy

Technology Characteristics

Data Aggregation

Fig. 2. TTF before OLAP capability

describing the health council planner's characteristicsprior to receiving access to OLAP technology (Fig. 2).We follow with a description of the changes after theimplementation of the OLAP interface. To illustrate thecontext, we include quotes from the health councilplanners that were obtained during the interviewprocess.

5.1. Individual characteristics

Workers at the health council (referred to as healthplanners) exhibit individual characteristics that onewould expect to find in knowledge workers. Knowledgeworkers are defined as “employees who apply their ownknowledge, acquired through experience and education,to develop new knowledge or apply existing knowl-edge” [9]. The characteristics identified from theinterview data were skills, experience, autonomy, andcreativity.

The individuals at the health council all have asignificant level of skills. They have master's degrees inhealth services administration and are supervised by anindividual that recently received a Ph.D. in PublicAffairs. Their work requires that they have a good levelof familiarity with word processors and spreadsheets, aswell as some basic usage of queries to PC databases.According to a health planner, “the more (computerskills we have) the better”. The same health plannerexplained that it is imperative to have effectivecommunication skills, “To talk to the person you aregetting data for, to clarify not only what you are going todo for them, but what they are looking for”.

Often these health planners apply their own experi-ence: “[With research for grants], I start at the beginning

Outcomes

Quantity of Grants Quality of Output

(adapted from Goodhue [11]).

1311M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

and work my way through, and I know from work I havedone before of a particular indicator that will work inthat particular situation.” They refine their workprocesses so they can be most effective: “I keep noteson ways I collected data on previous projects becausesometimes you need to go back because it may be such aunique thing they are asking for and you went throughsuch a convoluted way to get the data and its hard toremember, so if I want to get this same data for thispopulation, what did I do, how did I get thisinformation? Sometimes I learn too, that I can skipsome steps and still get to the same answer. So I wasteless time to get to my answer.”

Their tasks are not predefined and they haveautonomy in how they perform their work. From theinterviews, a recurrent theme is that these workersenjoyed a challenge, and were not hesitant to change: “Inever had something so complicated I gave up”. Whendescribing her job, a health planner explained, “I don'thave any constraints. No one tells me how to do it, theyjust want it [data and analyses]. They identify aproblem…Now we can go further and define it. Thesolution is more effective [because] I can further definethe problem.” During discussion about ideal toolcharacteristics the health planners displayed theirpreference for autonomy, “I want this [tool] to be anarea we have expertise in”, “[In response to developerwho suggested a less flexible, but simpler web interface]No! We can do that. I'd rather have the flexibility.”When talking about the interface “I prefer to [be able to]manipulate [the data] so I can get what I need”.

5.2. Task characteristics

Understanding the characteristics of healthcareplanning is important because task characteristics canaccount for up to 50% of the variance in performance. Ingeneral, the notion of task revolves around the activitiesor behaviors that individuals need to do to accomplish agoal. In this sense, task has been broadly defined as theactions carried out by individuals in turning inputs intooutputs [11]. When considering the tasks performed byemployees of the health council, it is apparent they fallinto the category of knowledge work.

The principal activities of a task that is classified asknowledge work revolve around “the acquisition,creation, packaging, or application of knowledge” [5].Acquisition entails the activities required to understandknowledge requirements, searching for the require-ments, and then preparing the knowledge for transfer toa requester or user. Creation consists of the researchactivities and creative tasks that are performed to

generate new knowledge. Packaging involves thepreparation and assembly of knowledge for consump-tion by a requester or user. Finally, application consistsof the activities that involve the use of existingknowledge in a situation. While the general activitiesare described in literature on knowledge work, thespecific tasks performed by knowledge workers tend tovary greatly from task to task, such that there is littleroutine or repetition and a greater emphasis on creativity[5,9].

Prior to having direct access to the data warehousethrough the OLAP interface, the tasks carried out by thehealth planners encompassed the characteristics de-scribed above, but fell predominately into two broadcategories: knowledge acquisition and knowledgepackaging. Their tasks often consisted of short-termresearch projects conducted on behalf of a diverse set ofclients from direct healthcare providers to philanthropicfoundations. As explained by a health planner: “Thehealth planner [position] involves pulling together othercommunity members and then planning a program or aprocess to fill a gap in the healthcare system”. Some ofthese tasks can be broadly categorized as knowledgeacquisition, where the planners are mainly collectingdata for their clients and carrying out some dataanalysis. A health planner described it as “pulling dataand analyzing it to find out where there is a need, who itaffects and how it affects them”. An example is whenthey were asked to investigate “what is happening tokids and kids health, how many kids are gettingemergency care” for a task force on children's health.Another example was a request from a senior resourcealliance: “We want all you have on senior health care”.They also received many phone requests on physicians'capacity, “ how many practicing physicians (that areseeing patients) do we have in the area – (that is) veryimportant to us – for malpractice and for healthplanning – for example there are abandoned plans tobuild an OB expansion because there are not enoughphysicians…there are many issues”.

Some of their tasks can be categorized as knowledgepackaging. They often receive requests from specialtygroups, requesting “everything that is happening withthe X population, broken down by country of origin,because there are disparities by that”. Another goodexample of knowledge packaging is a particular requestfor the Ryan-White Title 1 (funding to support healthand social services for low-income and un- or under-insured persons living with HIV/AIDS and theirfamilies) where they need to compare epidemiologyversus utilization data: “All of that data comes to me andI analyze it, chart it, and prepare presentations for their

1312 M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

committee meetings. In addition, they have asked for usto come up for the population that has AIDS whoseneeds are not met.”

5.3. Technology available to health planners

Technology is defined as the tools used byindividuals to carry out tasks [12]. Technology caninfluence performance outcomes to the degree that it isappropriately designed for the task and used by theindividual [6]. The primary difficulty in examiningtechnology to support knowledge work is the fact that“The abstract and unstructured inputs to and outputsfrom knowledge work processes make the application oftechnology more difficult” [5].

Prior to the availability of OLAP and data ware-housing technologies, the health council workers usedseveral data sources of aggregated data. In particularthey used hard copy reports printed from the CATCHdata warehouse, census bureau data, and data availablefrom the Florida Health Department, as well as reportsprovided by a local university. These sources of data

Fig. 3. A page from a CAT

were similar in that they had a fixed format: they werestatic, structured, and contained a fixed level ofaggregation. For example, the reports provided by theCATCH data warehouse were helpful for the healthcouncil workers (as previously described), but limited intheir flexibility. Fig. 3 shows an example page from aCATCH data warehouse generated report.

This particular report outlines avoidable hospitaliza-tion events due to Asthma attacks. It compares onecounty with similar counties across several years. Thisdata is helpful; however, it cannot be easily manipulated.For example, it is not possible to compare this indicatoracross races, or socioeconomic status. Therefore, thesehealth planners had sufficient tools and data available tothem, but the lack of flexibility did not allow them tofully leverage their skills and experience.

In summary, we would expect that to the degree thatthe technology supports the nature of the knowledgework performed by the health planning agency, andprovides flexibility to support the manner in which theagency's employees wish to approach and perform theirtasks, higher performance would result [2,7]. Although

CH hardcopy report.

1313M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

the health planners were productive and successful, itwas evident that due to the limitations inherent in thedata and tools (e.g., hardcopy CATCH reports, staticsources of information, limited integration between thesources of data and the tools needed for analysis), thesehealth council planners could not fully leverage theirindividual characteristics and abilities. Therefore, anytechnology applied in the context of the health councilmust be able to support tasks that require data to beaccumulated from diverse sources while providing theuser with the ability to utilize autonomy and creativity inhow they manipulate, analyze, and present data. Wefollow with a discussion of how the characteristics of thetasks, individuals and technology evolved after theintroduction of an OLAP tool.

6. Introducing OLAP functionality

The health planning agency being studied has usedthe CATCH methodology in the past and the associatedinitial hardcopy reports from the CATCH data ware-house have certainly had an impact throughout Florida.However, the implementation being studied in this papermarks the first time that true OLAP data warehouseaccess has been available.

We expected that although the health council hadtechnology that supported them in serving their clients,the introduction of a flexible OLAP data warehouseinterface would help them to improve their performanceby allowing them to better leverage their experience,

Fit

Task Characteristics

Knowledge Acquisition Knowledge Packaging Knowledge Creation Knowledge Application

Individual Characteristics

Creativity Experience Skills Autonomy Interpretation Judgment

Technology Characteristics

Data Aggregation Flexible Analysis Flexible Presentation

Fig. 4. TTF after users are g

skills, creativity and autonomy. A tool that is flexible inits ability to combine, manipulate, and present the datashould assist the health council knowledge workers infurther defining and refining tasks, as well as supportingvarious methods of exploring, analyzing and presentinginformation.

We use quotes from interview scripts, as well asexample OLAP queries, to explain and highlight theseemerging characteristics (Fig. 4). We begin byintroducing the new capabilities this technologygave to the health planners and how this allowedthem to utilize skills and expand their tasks andoutcomes.

6.1. Flexible analysis and presentation

One of the characteristics that typify OLAP toolsand data warehouse technologies is the capability toprovide data at many levels of granularity for multipleperspectives. Fig. 5 depicts an OLAP worksheet thatprovides county level data for various cancer mortalityindicators. This ability to traverse hierarchies, selectingappropriate levels of abstraction across multipledimensions, is a key capability that makes OLAPtools a natural fit for the tasks facing health planners aswell as other knowledge workers. Even at standardlevels, such as county health status indicators, theOLAP interface provides advantages over previoushardcopy or spreadsheet reports. First, the currenthardcopy reports have grown to upward of 700 pages,

Outcomes

Quantity of Grants Quality of Output Consulting Reputation

iven OLAP capability.

Fig. 5. County-level cancer mortality indicators.

1314 M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

making it difficult to justify new tables and multiplelevels of abstraction. Using online access, any numberof tables can be delivered at a wide range ofgranularities. In addition, there is no limit on thenumber of new health status indicators that can beintroduced, so the development of data cubes isongoing, driven both by the suggestions of the healthplanners and the developers. Another critical dimensionis time and much longer-term trends can be presented inan interactive fashion. In this case, the most usefulbenefit of OLAP access to the CATCH data warehousemight be the finer grained perspectives, such as zip codelevel analyses, that can be provided (as shown insubsequent sections).

A good example of how a finer grained perspectivewas useful was a particular request for the number oflicensed physicians by one of the data analysts.According to the health planners, “what they reallywant to know [is] how many physicians are practicing?You have to talk to them and have discussions on howit's going to be used, what you are really looking for andthere was no way to find out.” In this particular case, thehealth planners were able to change the focus from asimplistic perspective based solely on licensure data toan aggregated analysis using the hospital discharge data,ambulatory care data, and licensure data. Physicianlicensure data provides a snapshot in time with noinformation on actual practice patterns. By linking thedata sets, the health planners were able to explore trendinformation across a wide variety of procedure ordiagnostic categories. For example, Fig. 6 shows aquery requesting the number of unique physiciansperforming orthopedic surgery in the community ofinterest, along with the raw numbers and case load. Inthis case, the planners were able to identify that there is

growth in both the number of doctors and the averagecase load, probably due to a variety of factors such aspopulation growth.

The health planners indicated although they weresatisfied with the additional capabilities that OLAP toolprovided them, there were some difficulties. Theirprimary complaint is they occasionally have to use othertools for data manipulation, e.g. EXCEL. Theyunderstand, however, that it is not always possible tocombine certain data: “Because we are always bench-marking our county against other counties, against thestate, or against other communities it is a little bitdifficult when you want to look at all three withoutgoing from workbook to workbook and not all theworkbooks are in the same order. Because of the waysthey are calculated, not all indicators are calculated forthe county, the state, or the peer groups. But when I wantto look across the board for one indicator it's a lot ofwork, just busy work. You have to copy and paste, lineup all the indicators”.

6.2. Interpretation and judgment

The added flexibility provided by the OLAP toolallowed the health planners to leverage their individualskills while allowing them to exercise some new skills,in particular: interpretation and judgment. Interpretationbecame a large part of their work: “In the past I justexported it [acquired data]. Now, the knowledge of thepeople out there is not the same as ours, so I interpret,and they can use it”. “I now have the capability ofdigging deeper into the data than what they ask for, andprovide that information back to them.”

The requests from their clients are sometimesambiguous, and the health planners are now able to

Fig. 6. Physician practice patterns in orthopedic surgery.

1315M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

“cross-examine” their clients in order to interpret whatthey actually need. “That is another problem thatcomes up, people may ask for something, but they arenot asking for what they really want.” As healthplanner explained: “I talk to the requestor and I askwhat is it going to be used for, is it for a grant,because that really helps you to help them, understandexactly what it is they are looking for. Sometimes theydon't know!” “They have some pre-conceived picturein their head of what this is going to look like, butthere are limitations. Trying to communicate this hasbeen a real challenge”.

Interpreting the clients' needs and the data availablehas required the health planners to exercise moreindividual judgment than previously required: “This isthe first time that we have to be really careful to makesure how we are interpreting that data.” Recounting anexample of this a health planner stated: “… someonewants breast cancer indicators. They want to knowmaybe how bad the breast cancer in a county is and whatzip code is the worst. Because a lot of people that wantthe data don't know necessarily or understand how tointerpret the data once you give the numbers. So youhave to go in and interpret it so they don't misinterpret itonce you give them the numbers.”

They also express some frustration with their ownlevel of expertise. They make several references toneeding more training: “I don't have the experience togo in and see if you can do a query it's a little bit timeconsuming for me. Working with it, I have developedwhat I think it's the quickest process for me and youknow it gets better each time. I have discussed it with thedeveloper, he is thinking about what he can do”.

According to the developer, they have the mostflexibility possible with the data cubes that had beendesigned thus far. Their insistence on further trainingillustrates that the users were learning how to use thetool quickly and moving beyond its original capabilities.It is also becoming indispensable for their work. Thedevelopers comment that “it is really time to redesignand develop new data cube components in light of thecurrent experiences.”

6.3. Additional task characteristics

The flexibility and availability of the OLAP interfaceallowed health planners to accomplish new tasks whichcan be broadly categorized as: knowledge applicationand knowledge creation. In early interviews, theyanticipated that the OLAP interface would aid them bymore efficiently getting the right data from thewarehouse: “They have asked for us to come up withthe population that has AIDS whose needs are not met.Currently we have no way to come up with thatinformation, because we do not have access to thehospital inpatient data. I can't do my job; I can come upwith some sort of convoluted educated guess. But if Ihad access to the [detailed] data, I could come up with anumber that was more accurate.”

Knowledge application became possible because ofthe rollup and drill down capabilities which made itmuch easier for them to find data for planning purposes:“For example, if they identified a problem withcardiovascular disease, I can now go into the threedifferent communities we have designed and I can lookat which community, in the socioeconomic class, and

1316 M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

see if is it in the higher or in the lower. Before we had arate for the whole county, but now we can go and furtherdefine that. That makes planning a lot easier becauseyou are actually planning to solve the problem where itis. The solution is more effective. You can further definethe problem.”

Knowledge creation also emerges as indicated by oneof the health planners in this example: “[There is a]Health Initiatives group. There are 10 health indicatorsneeded. There are two groups: barriers to care and health[status] indicators. For barriers to care I had to researchhow long it takes for them to get into some sort oftemporary housing, what percentage of the populationspeak a language other then English, the unemploymentrate for that particular population in that particularcounty, the percentage bellow poverty level, the percentof uninsured (this is all county specific, by race orethnicity). Same is true for health indicators; I had to findcancer rates, infant mortality rates, asthma rates, obesityrates, diabetes rates, STD/HIV/AIDS rates, low birthweight rates, depression rates, rate of respiratoryinfections, chronic liver disease rates. What makes itdifficult is that I don't know what their goal is.”

Fig. 7 depicts an analysis based on communitiescompiled by the health planners, in this case based onHispanic population. The rate and trend information forchronic heart disease are shown for various Hispanicconcentrations. These synthetic communities are com-posed of non-contiguous zip codes that highlight issuesacross racial composition, income levels, or other criteriaselected by the health planners. Multiple perspectives aresupported through different community definitions,based on factors such as income, poverty, or agedistribution. The different community groupings weredefined by the health planners and implemented in thedata warehouse to provide additional OLAP aggregationopportunities.

Fig. 7. Chronic heart disease for synthetic com

6.4. Consulting and reputation

Although it is difficult to measure impact on thecommunity at such an early stage (many of the health-related projects run several years), it was apparent that theOLAP interface to the CATCHdatawarehouse hadmade adifference in the effectiveness of these workers as well aswhat role they played as health planners.When asked howthey would feel about not having OLAP capability: “Wecan't dowithout it now.We use them for almost everythingwe do. It would be really hard to go backwithout it. Before,we had just one mediocre source of data.”

It was articulated by the health planners that, as theywere able to acquire more data through the OLAPinterface, their roles began to change. Rather than datacollectors, they began acting more as consultants: “Thisis the first time we have to be really careful to make surehow we are interpreting that data. I used to give themdata straight, but now I can actually go in and say this iswhat we found, but when you look at this, and you lookat this, you know it kind of changes your focus. I am nowa better resource. I have tomaintain an unbiased position,I can't lobby for a certain position, but I can certainlyhave a discussion. Before somebody asked and I gave itto them, but now I have more information so I canenlighten them, but then it's their decision to take thosethings into account and go forward. We have thecapability of digging deeper into the data than whatthey ask for and provide that information back to them.”These deeper analyses often include drill downs tocommunity or even zip code level data. For instance, thelow birth weight mentioned above can be viewed by zipcode to pinpoint areas for intervention. Fig. 8 provides anexample query that shows a zip code perspective on lowbirth weight for a specific county.

In one particular case where the health planners actedas consultants, they were able to indicate areas with

munities based on Hispanic population.

Fig. 8. Low birth weight by Zip Co.

1317M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

special needs: “You can't just put a mobile mammogramin X or Y, because those people have insurance, they goto their physicians. But you can see that [a particulararea] has 3 times the rate then another zip code, then youknow you can put one there. Whereas if you only hadcounty data you can put all the units you want if it's notin the right place… Before, we would have had to rely onwhat people in the community said. But when you go fora grant you can't just say we think there is a problemhere. They always want hard data. [In the grant] theywere able to substantiate rates were high in that zip code.So they could bring mammograms to the right people inthe right zip code. People who really needed it.”

In one case they used the OLAP tool to define a moreappropriate level of granularity for a client: “one personwanted to do some kind of special oncology care forwomen: basically breast cancer, or cervical cancer. Sheasked for late stage cervical cancer by zip code.Unfortunately, we don't have that. Because the cervicalcancer rates are really pretty low, that once you breakthem down to zip codes the numbers are useless. Theyare too low to be significant, so if you have ‘1’ in a zipcode you really can't use it. I was able to give it to her[the data] by county, and by zip code for other cancers,but not for late stage cervical cancer.” This analysis wasapproached by defining communities within the countyas outlined above.

The tool allowed them to give more detailedinformation. For example “we were looking at schizo-phrenia, it has quadrupled in 3 or 4years. What iscausing this? Is it that more people are presentingthemselves? We know already that there are limitedmental health facilities; we have a real shortage here. Soit's not like there are more facilities being able to handlemore clients. Was it a change in insurance?” Changes inreimbursement policies, coding practices, or even theunderlying coding system itself can affect the healthcareutilization picture.

OLAP capabilities allowed them to provide data tosubstantiate other findings: “The health foundation didthis qualitative health assessment. It was a 135 questionsurvey, they over sampled in areas where they did nothave info on.” They found that alcohol related motorvehicle mortality was enormous. “The CATCH [datawarehouse] helped us see where it was astronomical.Was it at a particular age, socioeconomic [category], orzip code (maybe there is an intersection)?” Based on theinformation they could explore potential causes anddesign an intervention.

The impact to the community was described by thehealth planners using the reaction of the groups theyserved: “People are really happy with the data; they keepreturning and asking for more.” In a recent annualmeeting, it was noted that the Health council had beenreally instrumental. In particular, community clientswere satisfied with quick turn around and the fact that thehealth council actually had what they were looking for.“They could pull the data themselves, but I can pull allchronic diseases across zip code – before I had to do eachindicator – now I can do comparisons, benchmarking.”

They continue to receive constant positive feedbackfrom their clients. A measure of their improvingreputation is the amount of requests for data that theyreceive and that has been on the rise. Overall the healthplanners feel that they are being more effective: “Itsquicker, its more effective more efficient, gives a betterpicture.” They had a high level of trust on the data,although before disseminating it they always made surethey understood it: “I have to really understand the databefore I give it out, not that it's wrong, it's just adifferent perspective. Like for example if I were to giveout that information about schizophrenia I would haveto understand it, I can't just say, oh yeah schizophreniais rampant in Orlando, I have to substantiate that, I haveto find out why. You have to be really careful becausethis is all new and no one has ever had this before.”

1318 M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

Sometimes they still go to alternate data sourceswhen the data looks suspect: “Not that I mistrust it, butsometimes there are goofy things with the data. Forexample, for some reason for the year 2000 the rateswere just double what they were for the previous 2yearsand the following 2years. What caused this, are you sureit's right, because as soon as I go and present this datathey are going to ask what happened in 2000? Why ishis like that? We were not able to find out why–it wascorrect–maybe for that year they were counted differ-ently, maybe due to some piece of legislation.”

7. Conclusions and future work

The results of this field study illustrate how this typeof technology has the potential to make strongcontributions to performance, especially in the assess-ment and review of healthcare planning. The examplesprovided demonstrate how the availability of more datain an integrated data warehouse, combined with aflexible OLAP interface, allowed this agency to becomemore effective in serving the community.

This study provided an exceptional opportunity tostudy knowledge workers in a real life context. Thehealth planners display the characteristics outlined bythe literature on knowledge work [23]. They are allhighly skilled individuals with advanced degrees and asuperior skill set, whose tasks are complex and oftenrequire creativity in coming to a solution. Often thesehealth planners apply their own experience andconstantly refine their processes so as to improve theireffectiveness.

Several ideas emerged from observing the healthplanners interact with this malleable business intelli-gence technology. We observed that their individual andtask characteristics evolved, as did the outcomes. Astheir level of expertise with the OLAP tool increased,their job roles began to change. Rather than datacollectors, they began performing more as consultants.Before the implementation of this decision supportsystem, their job consisted of finding data and providingit to their customers. Using the OLAP interface, theybegan providing their users with highly detailed data,along with interpretations and descriptions. They alsoused individual judgment to advise their clients whichdata they really needed. The health planners were nolonger only acquiring and packaging knowledge, butcreating and applying it as well. As a result, theirreputation among their peers and clients increased,escalating the requests for data from this particularhealth council. Although we have not had time to collectexact data, in a recent conversation with the supervisor

of this group, we were told that the council now has afull time employee whose main responsibility isprioritizing and selecting data requests for the group.

It is important to note that because this study isconducted within a single health council, generalizationof the results to other organizational settings is limited.However, this implementation has aroused the interestsof other health councils in the State of Florida, as well asin other states. These additional sites can be used tofurther validate and refine our study [25]. While wewould like to think these results are generalizable to anyknowledge worker context, given the nature and uniquecontext of healthcare planning, our results are likelymost generalizable to that context. Additional researchin different contexts would help determine to the degreethat these changes occur in other environments.

Our purpose was to study the impact of theimplementation of OLAP access to a data warehousewithin a regional health planning agency in the State ofFlorida. Specifically, we were interested in the degreethat this particular technology fit the nature of theknowledge work performed by the health planningagency, and whether it provided the flexibility to supportthe manner in which the agency's employeesapproached and performed their tasks. Thus, a qualita-tive approach was preferred to a statistical generaliza-tion. Because of our approach, other potentiallimitations are the biases and preconceptions that theresearchers had when interpreting the data. In order tominimize this possibility, interviews scripts weregenerated a priori using Task-Technology Fit Theory(TTF) as our framework. A template was created to codethe interviews and multiple raters were used tocorroborate patterns and findings in the data. However,additional research in this area is necessary tocorroborate our findings.

7.1. Implications for research and practice

In this research we did not utilize TTF as aninstrument with which to measure constructs, but rather,to guide us in understanding the changes that businessintelligence technology such as OLAP can bring about,and whether it can improve the outcomes for those whouse it. We believe a main contribution of this work is theidentification of the new individual, task and theoutcome characteristics that emerged during this study.These characteristics suggest new conceptualizations ofthe TTF constructs that may be more applicable in thedomain of flexible decision support technologies such asOLAP. While TTF has offered significant explanatorypower, an expansion on the TTF constructs may be more

1319M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

insightful for these types of technology. Our researchgroup has begun work in this area in a more controlledstudy with the purpose of further theory development.

Research on these technologies can benefit fromdifferent perspectives. Through the use of a longitudinalstudy at an individual level of analysis, we were able toobserve the transformation of these individuals and theirtasks. These tools are evolutionary, with the changesbeing very much driven by the clients. The individualsthat use these tools also change over time. Longitudinalresearch allows many observations over time that enablechanges in use, outcomes, and even individual char-acteristics to be captured. Future research in this areashould consider various perspectives and approaches torobustly capture the many changes and evolutionarynature of these technologies.

Finally, understanding the development process forthese technologies is an area for future research. Thefindings from these types of research can provide usefulinsight on the development of these tools and themethod of their implementation (e.g., facilitation andtraining methods) to increase the likelihood of success-ful adoption and use of these technologies. Researchcould also focus on identifying and refining the conceptof performance as it relates to healthcare planning, aswell as documenting some of the important impacts thistype of technology, as used by health planners, can haveon the communities served by these councils.

The results of this research also identify implica-tions for practitioners in this area. Through the courseof this research, we observed a high level ofengagement between the knowledge workers and thedevelopers of the OLAP interface. From the begin-ning, this was a joint development effort, and one thatis still ongoing. In fact, from the first training session,the users expressed interest in being directly involvedin the development of the tool. The developerscreated data cubes that were often guided by requestsfrom the health planners. For example, one of theirfirst requests was to have the ability to track healthindicators against other states and the ability tobenchmark against regions that are similar in otherstates. They were not intimidated by the complexityof the tool, in fact the developers suggested “If whatyou did was mostly routine, we can build a webinterface that is simple to use but not with muchflexibility”. The user is a complex tool: “I am gladabout that, I don't want any old Joe to come in andbe able to use this, I would like this to be an area wehave expertise in.” They have developed a goodrelationship with the developers. This was important,since development was essentially ongoing. “I can

pick up at any time and call the developers and sayOK, what's the deal with this, or what does thismean, or, is this really OK.”

Communication with the developers was frequent,and the developers were very accommodating in rollingout more cubes or views as requested by the healthplanners. Yet, sometimes the health planners expressedsome frustration with their level of expertise, highlight-ing the importance of the correct balance of training andthe timing the introduction of new functionality.Practitioners in this area must acknowledge the higherlevels of interaction between users and developers thatshould exist for these tools to be successful. Further-more, a high degree of interactive planning betweenusers and developers would be helpful, to balance thedelivery of new capability with training on thatcapability. We observed the OLAP implementation tobe a highly interactive dialog between user anddeveloper, which certainly had a positive impact onthe successful implementation (to this point) of this tool.

In conclusion, we found that on-line analyticalprocessing (OLAP) is a powerful tool that can supportknowledge workers in healthcare planning. The resultsof this field study show that the additional capabilities ofOLAP supported the tasks and leveraged the skills ofhealthcare planners to generate positive outcomes on thelevel of service (and services) that they could provide totheir community. We observed that this type oftechnology has the potential to make strong contribu-tions to outcomes and performance, especially in thedata-intensive environment of healthcare planningassessment and review.

Acknowledgement

We would like to thank our anonymous reviewers fortheir invaluable comments and suggestions and theworkers at the health council for their endless patienceand enthusiasm.

References

[1] I. Benbasat, D.K. Goldstein, M. Mead, The case research strategyin studies of information systems, MIS Quarterly 11 (3) (1987).

[2] L. Chidambaram, B. Jones, Impact of communication mediumand computer support on group perceptions and performance—acomparison of face-to-face and dispersed meetings, MISQuarterly 17 (4) (1993).

[3] E.F. Codd, S.B. Codd, C.T. Salley, Providing On-Line AnalyticalProcessing to User-Analysts: An IT Mandate, E.F. Codd andAssociates, 1993.

[4] A. Datta, H. Thomas, The cube data model: a conceptual modeland algebra for on-line analytical processing in data warehouses,Decision Support Systems 27 (3) (1999).

1320 M.C. Tremblay et al. / Decision Support Systems 43 (2007) 1305–1320

[5] T.H. Davenport, S.L. Jarvenpaa, M.C. Beers, Improvingknowledge work processes, Sloan Management Review 37 (4)(1996).

[6] G. Desanctis, G.R., A foundation for the study of group decisionsupport systems, Management Science 33 (4) (1987).

[7] G.W. Dickson, G. Desanctis, Understanding the effectiveness ofcomputer-graphics for decision support—a cumulative experi-mental approach, Communications of the ACM 29 (1) (1986).

[8] N.F. Doherty, G. Doig, An analysis of the anticipated culturalimpacts of the implementation of data warehouses, IEEETransactions on Engineering Management 50 (1) (2003).

[9] P.F. Drucker, Post-Capitalist Society, Harper Collins, New York,NY, 1993.

[10] K. Glassey, Seducing the end user. Association for computingmachinery, Communications of the ACM 41 (9) (1998).

[11] D.L. Goodhue, Understanding user evaluations of informationsystems, Management Science 41 (12) (1995).

[12] D.L. Goodhue, R.L. Thompso, Task-technology fit and individ-ual-performance, MIS Quarterly 19 (2) (1995).

[13] T. Hoffman, Conference attendees: CRM initiatives may misstheir marks, Computerworld 35 (9) (2001).

[14] R. Kimball et al., in: W.C. Publishing (Ed.), The Data WarehouseLifecycle Toolkit, John Wiley & Sons, Inc., New York, 1998.

[15] N. King, Template analysis, in: G. Symon, C. Cassell (Eds.),Qualitative Methods and Analysis in Organizational Research,Sage Publications, London, 1998.

[16] N.-S. Koutsoukis, G. Mitra, C. Lucas, Adapting on-lineanalytical processing for decision modeling: the interaction ofinformation and decision technologies, Decision SupportSystems 26 (1) (1999).

[17] R.G. Little Jr., M.L. Gibson, Identification of Factors Affectingthe Implementation of Data Warehousing, 32nd Hawaii Interna-tional Conference on System Sciences, 1999, Hawaii.

[18] S. Sarawagi, R. Agrawal, N. Megiddo, in: I.A.R. Center (Ed.),Discovery-driven exploration of OLAP Data Cubes, San Jose,California, 1998.

[19] M.J. Shaw, et al., Knowledge management and data mining formarketing, Decision Support Systems 31 (1) (2001).

[20] B. Shin, An exploratory investigation of system success factors indata warehousing, Journal of the Association for InformationSystems 4 (2003).

[21] A.L. Strauss, J.M. Corbin, Basics of Qualitative Research:Techniques and Procedures for Developing Grounded Theory,2nd ed., Sage Publications, Thousand Oaks, 1998, p. xiii, 312.

[22] J. Studnicki, et al., Comprehensive assessment for trackingcommunity health (CATCH), Best Practices and Benchmarkingin Healthcare 2 (5) (1997).

[23] J.D. Wells, T.J. Hess, Understanding decision-making in datawarehousing and related decision support systems: an explana-tory study of a customer relationship management application,Information Resources Management Journal 15 (4) (2002).

[24] W.F. Whyte, K.K. Whyte, Learning from the Field: A Guide fromExperience, Sage Publications, Beverly Hills, 1984, p. 295.

[25] S.R. Williams, R.L. Wilson, Group support systems, power andinfluence in an organization: a field study, Decision Sciences 28(4) (1997).

[26] B.H. Wixom, H.J. Watson, An empirical investigation of thefactors affecting data warehousing success, MIS Quarterly 25 (1)(2001).

[27] R. Yin, Case study research—design and methods, in: L.Bickman (Ed.), Applied Social Research Methods, vol. 5, SagePublications, Thousand Oaks, 2003.

Monica Chiarini Tremblay is a DoctoralCandidate at the University of South Florida.She received an MS in Information Systemsdegree from the University of South Floridaand a BS degree in Industrial and SystemsEngineering from the University of Florida.Prior to pursuing her PhD, Ms Tremblayworked as a systems analyst. Her researchinterests include decision support systems,data warehousing, knowledge discovery, dataand text mining.

Robert M. Fuller is an Assistant Professor inthe Information Systems and DecisionSciences Department in the College ofBusiness Administration at the Universityof South Florida. His research interestsinclude decision support systems, collabora-tive technologies, and computer-mediatedcommunication. Dr. Fuller received his PhDin Management Information Systems fromthe Kelley School of Business at IndianaUniversity.

Donald J. Berndt is an Associate Professor inthe Information Systems and DecisionSciences Department in the College ofBusiness Administration at the Universityof South Florida. His research interestsinclude data warehousing, knowledge dis-covery, data mining, and health informatics.Dr. Berndt received a PhD in InformationSystems from the Stern School of Business atNew York University.

James Studnicki is a Professor of HealthPolicy and Management at the University ofSouth Florida College of Public Health. Hisresearch interests include measuring thehealth status of communities, evaluatingalternative treatment outcomes, and studyingthe influence of managed care penetration onthe utilization and quality of health services.Dr. Studnicki received a ScD from JohnsHopkins University.