An assessment ofroutine telepsychiatryservices

Jennifer Simpson, Sandra Doze,Douglas Urness, David Hailey,Philip Jacobs

November 1999

Alberta Heritage Foundation for Medical Research

HTA 20

An assessment ofroutine telepsychiatry

services

Jennifer Simpson, Sandra Doze,Douglas Urness, David Hailey,

Philip Jacobs

November 1999

© Copyright Alberta Mental Health Board and Alberta Heritage Foundation forMedical Research, 1999

Additional information and comments relative to this report are welcome andshould be sent to the authors at:

Alberta Mental Health BoardP.O. Box 4422Ponoka, AlbertaT4J 1S1

or to:

Director, Health Technology AssessmentAlberta Heritage Foundation for Medical Research3125 ManuLife Place, 10180 - 101 StreetEdmonton, Alberta T5J 3S4CANADA

Tel: 780-423-5727, Fax: 780-429-3509

ISBN 1-896956-25-4

Alberta's Health Technology Assessment program has been established underthe Health Research Collaboration Agreement between the Alberta HeritageFoundation for Medical Research and the Alberta Health Ministry.

i

Acknowledgments/organizationThe Alberta Mental Health Board and the Alberta Heritage Foundation forMedical Research gratefully acknowledge the contributions of the followingpeople to the telepsychiatry assessment project:

Telepsychiatry Site Coordinators:Wainwright: Cheryl Huxley, Health Centre CoordinatorDrumheller: Marion Ewing, Registered Psychiatric NurseRocky Mountain House: John Jackson, Assertive Outreach WorkerEdson: Karen Conover, Community Mental HealthHinton: Holly Day, Community Mental Health TherapistFort Vermilion: Michelle Dick, Mental Health TherapistHigh Level: Annie Pesht, Mental Health TherapistDrayton Valley: Connie Stuhl, Supervisor, Patient RegistrationWetaskiwin: Carol Potts, Unit Manager, General Ward

(Medical Floor)Mayerthorpe: Eric Schocat, Mental Health TherapistSt. Paul: Colleen Blanchette, Secretary, Psychiatric Unit

Evaluation Support:Margo Dowling SecretaryJo-Anne Gould Secretary to the Chiefs of ServicePatti Pugh Videoconferencing SupportGwen Bentley InterviewerAlison Kerik Research AssistantChristine Walters Research AssistantErin Hansen Research AssistantSerge Clapa Real Time TechnologiesKimbal Raymond Information Systems

ii

Psychiatric Consultants:Dr. Paul DarlingtonDr. William DelanyDr. Keith GibsonDr. Shawn Gray (Brain Injury Rehabilitation)Dr. Doug HuberDr. Majeed KhatriDr. Christina KorsakDr. Roland LynchDr. Kevin MaloneDr. Blair McCurrieDr. Bernard MurphyDr. Fergus O’CroininDr. George Rosenkranz

Crossroads Regional Health Authority:Alison Gorrell: Project Officer, Crossroads Regional Health Authority

Alberta Mental Health Board:Ken Sheehan: Chief Operating Officer, Facilities and Programs, Alberta Mental

Health BoardSharlene Stayberg: Administrative Director, Alberta Mental Health Board,

Telemental Health

iii

ContentsAcknowledgments/organization ................................................................................... i

Summary ........................................................................................................................... 1

Foreword ........................................................................................................................... 3

PART I: Routine operation of telepsychiatry

Assessment Areas ............................................................................................................ 6

Acceptance and effects on the health care deliveryincluding accessibility, quality and cost .............................................................. 6

Clinical performance .............................................................................................. 6

Perspective of consumers, family physicians and community ........................ 6

Development of a minimum data set................................................................... 6

Methodology..................................................................................................................... 7

Survey instruments................................................................................................. 7

Qualitative interviews............................................................................................ 8

Focus group ............................................................................................................. 9

Telepsychiatry outcome monitoring system....................................................... 9

Limitations and Changes to the Evaluation Process................................................. 12

Evolution of the TOMS database and use of TOMS in study analysis ......... 12

Approval process .................................................................................................. 13

Status of information system............................................................................... 13

Benchmark study .................................................................................................. 13

Results.............................................................................................................................. 14

Utilization............................................................................................................... 14

Consumer characteristics ..................................................................................... 20

Assessment of equipment.................................................................................... 24

Acceptance ............................................................................................................. 26

Cost Analysis ......................................................................................................... 28

Impact on Health Services Delivery............................................................................ 32

Other services provided by the Alberta mental health services .................... 32

iv

Outcomes................................................................................................................ 35

Satisfaction ............................................................................................................. 36

Discussion ....................................................................................................................... 38

Tables and Figures:

Table 1: Pilot study sites............................................................................................... 4

Table 2: TOMS data elements.................................................................................... 11

Table 3: Response rate for evaluation tools............................................................. 12

Table 4: Number of consultations per client ........................................................... 18

Table 5: Most common reason for consultations.................................................... 18

Table 6: Referrals by general practitioners.............................................................. 19

Table 7: Cancellations of consultations.................................................................... 19

Table 8: Profile of telepsychiatry consumers pilot versus routine....................... 21

Table 9: Consumer perceptions related to appropriateness of technology ........ 25

Table 10: Equipment problems during routine operation ...................................... 26

Table 11: Consumers perception from telephone interviews................................. 27

Table 12: Cost data for psychiatric consultations..................................................... 29

Table 13: Telepsychiatry consults and admission to Alberta Hospital Ponoka... 34

Table 14: Wait time for appointments in telepsychiatry ......................................... 35

Table 15: Results of physicians survey....................................................................... 37

Figure 1: TOMS data base............................................................................................ 10

Figure 2: Completed consultations by quarter ......................................................... 14

Figure 3: Completed consultation by pilot sites April 1, 1996 to March 31, 1999.......................................................................................... 15

Figure 4: Percentage of initial and follow-up consultations ................................... 16

Figure 5: Number of consumers seen only only compared to the total number of completed consults ................................................... 17

Figure 6: Percent of consumers who have one versus many visits ....................... 17

Figure 7: Percent of individuals using telepsychiatry by age group..................... 20

Figure 8: Provisional diagnosis at the time of the consultation ............................. 22

Figure 9: Consumers usual living arrangement at time of initial consult ............ 23

Figure 10: Profile of consumers from physician referral form ................................ 24

v

Figure 11: Assessment of equipment during routine use ........................................ 25

Figure 12: Cost comparison travelling consult and telepsychiatry ........................ 31

Figure 13: Referral to mental health clinics ................................................................ 33

Figure 14: Follow-up recommendations identified in the consultation report..... 34

PART II: Expansion of Telepsychiatry Services

Introduction .................................................................................................................... 42

Objectives and Methodology .............................................................................. 43

Results.............................................................................................................................. 45

Utilization............................................................................................................... 45

Consumer characteristics ..................................................................................... 47

Assessment of equipment.................................................................................... 47

Discussion ....................................................................................................................... 48

References ....................................................................................................................... 49

Tables and Figures

Table 16: Additional sites in the telepsychiatry service .......................................... 42

Table 17: Project description........................................................................................ 44

Table 18: Consultations per month by site ................................................................ 45

Table 19: Consumer profile.......................................................................................... 47

Figure 15: Completed consultations by month.......................................................... 46

Figure 16: Type of service area from July 24, 1998 to March 31, 1999.................... 46

Figure 17: Clients satisfied with the quality of the room, sound ease of use and picture ............................................................................... 48

1

Summary• The results of an assessment of a Telepsychiatry service beyond the six-month

pilot stage into routine operation suggest that the provision of psychiatricconsultations over video technology is a viable option in areas with limitedpsychiatric resources.

• Reduced wait time for an appointment and reduced travel time continued tobe identified as the major benefits to the consumer.

• The rate of 17 to 20 consultations per month established during the pilot stagecontinued through the assessment period with a total of 546 consults overtwo years.

• The population served during the assessment period was consistent with thefirst six month pilot period; the majority were female with a mean age of 40.Depression was the most common diagnosis (28%).

• The results of the pilot evaluation showed high satisfaction and acceptanceamong general practitioners, consumers and psychiatric consultants. Thishigh satisfaction continued during routine operation with an overallsatisfaction rate of 89%.

• While equipment problems were noted in 17% of all consultations in thesecond year, they did not seem to affect acceptance of the technology.

• A cost analysis comparing the costs of a consultation provided by a visitingpsychiatrist and those of telepsychiatry found a break even point of 350consultations a year. However, when use of the videoconferencing networkfor administrative and other meetings was taken into account, the break evenpoint was 224 consultations a year, substantially below the actual utilizationof telepsychiatry.

• If the consumer's perspective is considered, telepsychiatry provides savingsfor those individuals who would otherwise have to travel to a major centre tosee a psychiatrist. These savings might be of the order of $200 perconsultation.

• The utilization of the telepsychiatry service would appear to have beendetermined by a combination of factors including: the availability ofpsychiatric expertise locally; distance of patients from psychiatrists; otheravailable methods for accessing psychiatric advice; and integration with othermental/ general health services.

2

• Thirty-eight percent of those who received consultations were referred tolocal mental health services. The majority had not previously received mentalhealth services. This would suggest that the long term impact oftelepsychiatry may mean increased demand for community mental healthservices.

• The need to consider the use of telepsychiatry for emergency assessment andthe need for targeted marketing strategies were identified as common themesand areas of concern for communities, consumers and general practitioners.

• An integral part of the routine assessment was the development of theTelepsychiatry Outcome Monitoring System - a data base integratingadministrative and outcome information. This initial work should provide avaluable basis for the further refinement of outcomes information.

• During the assessment period the telepsychiatry service was expanded toinclude six additional sites. Limited information suggests that these sites maynot be as heavily utilized in the short term as the sites that participated in thepilot project.

3

ForewordA telepsychiatry pilot project, in which Alberta Hospital Ponoka was linked withfive general hospitals, was implemented in June 1996 as a response to both thescarcity of psychiatric resources in rural Alberta and increased emphasis oncommunity-based care. Details of the assessment of the pilot project have beengiven in earlier reports (4, 5).

The five sites chosen included three sites located more than 200 kilometres fromEdmonton, one site 138 kilometers from Calgary and one 80 kilometres fromEdmonton (Table 1). Sites had previously had access to a traveling psychiatristthrough the local mental health clinics. Information collected during the pilotevaluation showed that waiting times for a psychiatrist, either through themental health clinics or by referral to a city, were more than one month. Accessto a local traveling psychiatrist was determined solely by manpower availability.In the case of one site, a psychiatrist was not available for extended periods oftime. The only option for individuals was to travel to the city or be admitted tothe psychiatric treatment centre.

The telepsychiatry service provides psychiatric consultations to individualsbased on a referral from general practitioners. Referrals for service are eitherprovided directly to the telepsychiatry service or, depending on thecircumstances, may be forwarded to the service by the site coordinator or amental health therapist. Recommendations are then provided directly to thegeneral practitioners who arrange or provide follow-up as required. TheTelepsychiatry Service Director screens the initial referral for suitability.

Both the qualitative and quantitative information gathered during the pilotevaluation suggested that the use of video conferencing equipment forpsychiatric consultations should be considered when planning for an integratedcommunity-based mental health service. The information presented in theTelepsychiatry Pilot Project Evaluation was instrumental in the decision toexpand the project to eleven sites (5).

The present report presents data from the telepsychiatry program subsequent tothe pilot project. The assessment of telepsychiatry beyond the pilot project stagehad two primary focuses: a) to develop and document information collectionsystems initiated during the pilot project evaluation, and b) to assess theoperation of a routine telepsychiatry service.

Part I of this report presents information on the routine operation oftelepsychiatry at those sites that participated in the pilot project, pluspreliminary work completed on the inclusion of SF12 and EuroQol as measuresof clinical performance.

4

Part II focuses on the assessment of the experience of the pilot sites. Also, someinitial information on the six additional sites introduced during the assessmentperiod is presented.

Table 1: Pilot study sites

Centre Drumheller Edson DraytonValley

Wainwright Wetaskiwin

Population 7958 11,075 11,247 6,362 14,942Distance fromEdmonton

138(Calgary)

200 214 206 80

Source: Statistics Canada, 1996

PART I:

Routine operation of telepsychiatry

6

Assessment Areas

Acceptance and effects on the health care delivery includingaccessibility, quality and cost.Cost - The Pilot Project Evaluation provided an initial matrix analysis of thesocietal cost/benefit analysis from the perspective of the consumer, generalpractitioner and psychiatric consultant. Cost variables reviewed includedequipment set-up, manpower (including psychiatrist time), telephone chargesand cost to the consumer. The routine assessment will continue to consider costto providers and consumers.

Technical Quality - Systematic collection and presentation of data on equipmentand network reliability, quality of sound, picture, the interface between theequipment, the psychiatrist and the clients (for example: visually disabled orparanoid clients), and the service provided.

Accessibility/timeliness - As the demand on the system increases it will beimportant to continue to track wait times for appointment, length of time ofconsultation, utilization by site and physicians, and the profiles of repeat users.

Clinical performanceIn preliminary work undertaken during the present project a more completereview has been undertaken of the information provided by the familypractitioners on referral and the content of the psychiatric consultations. Clinicalperformance will be assessed through client satisfaction, health status andquality of life measures.

Perspective of consumers, family physicians, and communityContinued emphasis will be placed on obtaining information from the users ofthe system on the telepsychiatry service and the use of video conferencingtechnology.

Development of a minimum data setDuring the project, the project team developed a minimum data set fortelepsychiatry services, that can be incorporated into future routineadministrative practice.

7

Methodology

Survey instrumentsSurvey instruments developed during the pilot project (4) were used to permitcomparison between the routine and the pilot evaluation.

In order to solicit more meaningful data, the following enhancements weremade:

• the physician survey was modified to gather slightly different information;

• due to the limited number of technology issues identified during the pilotproject, consultants were no longer required to complete a separatetechnology assessment card;

• given few service providers were actually involved with the consultation, theservice providers’ form was not continued;

• attempts to capture systematic information on the recommendations of apsychiatrist resulted in the introduction of a therapeutic recommendationform.

Information for inclusion in the database and use by the evaluation included:

• Consultant’s Log Form: The psychiatrist completed a manual log after eachconsultation. Information recorded included time of consultation, equipmentproblems and type of consultation.

• Client Satisfaction Questionnaires: After a consultation was completed, thesite coordinators administered a satisfaction questionnaire to the client. Theclient was asked to complete the questionnaire and forward it to the sitecoordinator either in person or by mail. The site coordinator forwarded thequestionnaires to database support staff at Alberta Hospital Ponoka.

If the evaluation forms were not completed and returned within two weeks,the site co-coordinator was asked to reinforce to participants the importanceof the evaluation.

• Consultant’s Letter to the GP: The psychiatrist, following the consultation,completed the letter to the GP. This information is forwarded to the referringphysician. A copy of the letter is kept in the patient’s file and stored inclinical records. The Data Manager reviewed each letter and extractedinformation to the database.

• Quality of Life Questionnaire: During the study, preliminary work was doneon quality of life measures. Before the consultation, the site coordinatorsadministered a combined EuroQol and SF12 quality of life questionnaire to becompleted by the client.

8

• Therapeutic Recommendation Form: Following the consultation, thepsychiatrist completed a form outlining his/her therapeuticrecommendations (undertaken for part of the assessment project).

• Physician Survey: Questionnaires were sent to all physicians who hadreferred to telepsychiatry, regardless of how often. Physicians were giventwo weeks to respond and prompted with a telephone follow-up. No attemptwas made to track who had been sent forms. Several questionnaires fromphysicians were received too late for inclusion in the analysis.

• Referral for Consultation: The Referral for Consultation form continued tobe revised by the service administration. Information collected on the formincluded: presenting complaint and reason for referral; past psychiatrichistory; current medication; current medical condition; social/familybackground; and family psychiatric history. Referral forms were faxeddirectly to Alberta Hospital Ponoka or to the site co-coordinators who in turnfaxed them to the hospital.

Qualitative interviewsTelephone interviewConsumers were requested to indicate in writing whether they would beinterested in a telephone interview. Individuals that signed an agreement werecontacted by one of three separate interviewers.

Telephone surveys lasted about 15 minutes. Interviewers were instructed not toengage the participants in discussions on their illness or life situation apart fromthe structured questions.

All interviewers were instructed to attempt to call each person that consented toan interview at least three times.

Interviews with Site Coordinators

At the end of the routine assessment the Project Evaluators interviewed the sitecoordinators and one Chairman of the Regional Mental Health AdvisoryCommittee by videoconference. A free flowing discussion took place after thedata had been presented. Questions were open-ended and site coordinatorswere invited to make whatever comments they desired.

Consultant Interview

The Project Evaluators made themselves available for an interview to thepsychiatric consultants in a lunch hour meeting. Due to holidays, etc. only twopsychiatrists that had used the service attended.

9

Focus groupA focus group was held with the mental health clinic staff of the CrossroadsRegional Health Authority. A total of 13 staff attended.

Telepsychiatry outcome monitoring systemThe Telepsychiatry Outcome Monitoring System (TOMS) was developed, usingMicrosoft Access 97, for use by researchers, site coordinators, database managers,database programmers and consulting psychiatrists. It provides essentialinformation regarding the operation and definition of the TelepsychiatryOutcome Measurement System (TOMS) database.

TOMS contains client demographics, referral information, consultation status,satisfaction, equipment and environment problems and outcome measures on allclients that have been referred to the Telepsychiatry Service (Table 2).

The process for the collection of information is shown in Figure 1.

10

Figure 1: TOMS data base

ReferringPhysician

Patient

Site Coordinator

AHP AdministrativeSupport

AHP DatabaseSupport

Psychiatrist

Collects referral formfrom GP and forwardsit to AHP adminsupport

AdministersSatisfaction andQuality of Lifequestionnaires attime of consult

Collectsreferral formfrom Sites

Collects satisfactionand Quality of Lifeforms

General Practitionerprovides ReferralForm

Consultant’s letterto the GPforwarded to thePhysician

Completes Satisfactionand Quality of Lifequestionnaires

ForwardsSatisfaction andQuality of LifeQuestionnaires todatabase support

Track and scheduletelepsychiatry appointmentswith the Tracking Sheets

Data is entered intothe TOMS

Reviews referralform andassessesappropriatenessof request

Complete the Consultant’sLog form and theTherapeuticRecommendations form

Diagnosis andtreatmentprescribed inLetter to the GP

11

Table 2: TOMS data elements

Access – Utilization Quality – Equipment Assessment

• Referring physician• Reason for referral• Other service providers involved• Agencies involved• Service requested• Site• Booking information• Initial/Follow-up consultation• Service provider site

• Satisfaction with room, sound, ease of use,picture

• Appropriateness of technology forapplication

• Technology problems• Interaction with consultant

Client Characteristics Clinical Performance

• DOB• Gender• Employment status• Marital status• Patient location at time of referral• Living arrangements• Presenting complaint• Psychiatric history• Medical history• Quality of Life (EuroQol and SF12)

• Overall satisfaction• DSM - IV Diagnosis at time of consultation

including Global Assessment of Function• Quality of Life at one month post

consultation (EuroQol and SF12)• Medication changes• Psychotherapies recommended• Follow-up recommendations

12

Limitations and Changes to the Evaluation ProcessResponse rates for different evaluation tools used in the study are shown inTable 3.Table 3: Response rate for evaluation tools

Tools Rate

Consumer Questionnaires Pilot Period Response rate 69/91 (76%)Routine Period Response Rate 230/379 (62%)

Telephone Follow-up Interviews Consumers were phoned if they signed a consent agreeingto a follow-up telephone interview. 32 interviews during thepilot and 31 during the routine assessment. This representsa total of 14% of all those that received a consultation.

Physician Surveys A total of 24 surveys, 20 would had used the service werereturned from physicians in the pilot site areas out of apossible 74 referring physicians for a response rate of 32%.This is slightly less than the pilot evaluation.

Interviews with SiteCoordinators and other

A total of five people participated in the interviews.

Interviews with PsychiatricConsultants

Interviews with three psychiatric consultants were completed

Focus Group Mental HealthClinic

One group 13 participants

Evolution of the TOMS database and use of TOMS in studyanalysisDuring the assessment period the survey instruments and data collectionprotocols continued to evolve with the telepsychiatry service. The result of thisapproach was that at the end of the routine assessment period there werecapability problems with some of the variables. This was somewhat overcomeduring the analysis stage by re-coding of data.

Data variables in the TOMS database (during the defined routine use study) thatremained consistent with the pilot data based were merged with the pilotdatabase using SPSS. Re-coding was completed where necessary to ensureconsistency and comparability. Percentages and chi-squared values whereappropriate were used in the comparative analysis. Variables that were notdirectly comparable were reported separately. Emerging variables beingdeveloped during the routine phase, but not consistently used during routineuse, were omitted from the analyses.

13

During this period of development the TOMS database had not been subject to aPrivacy Impact assessment. A decision was made at that time that thesatisfaction questionnaire completed by the consumers at the time of theconsultation would not be linked to the individual files.

Approval processIn the period from the end of the initial pilot project evaluation in March 1997and the end of the assessment period there was a restructuring of the AlbertaMental Health Board and a reorganization of community clinic systems.Although not unusual, the result was that the researchers underestimated theamount of time required to receive approval for the various stakeholdersinvolved in the introduction of the SF12 and the EuroQol. In addition, a processto collect comparable information on clients seen and satisfaction with face-to-face consultations (the benchmark study) was not implemented until April 1999.

Status of information systemsDuring this time period significant effort was expended on the development of acommon information system to serve the needs of the community clinic system.The successful implementation of such a system would have supplied some ofthe data on waiting times for service, etc. that were identified as limitations in thepilot evaluation. This system is still in the development stages resulting insimilar data limitations and the reluctance of the clinicians to adopt any interimdata collection.

At the beginning of the routine assessment, information on the use of psychiatricservices in the selected communities was requested from the provincialgovernment. At this time the method for release of data to health serviceproviders was under review and the issue was not resolved in the study period.

Benchmark studyOne limitation of the Pilot Project Evaluation was that results are notbenchmarked against a similar service. In April 1999 a benchmark study wasinitiated which will continue for several months. The aim of the benchmarkstudy is to provide a context for interpreting tele-mental health results byevaluating two similar services provided in mental health clinics with thetelepsychiatry system in both the pilot and expanded network sites.

The data collected will include: patient characteristics (demographics, psychiatrichistory and reasons for referral), processes of care (wait times, consultationlengths), patient satisfaction and patient outcomes (quality of life before and afterthe consultation).

14

Comparable to the Pilot Project Evaluation, this study is designed to promoteprogram evaluation and to facilitate quality improvement efforts. Informationobtained will help increase understanding of telepsychiatry service evaluationresults, processes and methods.

Results

UtilizationNumber of consultationsDuring the period of assessment of routine operation, 24 months, a total of 548consultations were completed representing 379 people. The number ofconsultations leveled out with an average of 23 consultations per month. Thedecrease in the number of consultations in quarter four of routine operation canbe explained by psychiatrist availability, as the psychiatrist who was the majoruser of the service was away on holidays during that period (Figure 2).

Information presented at the end of the pilot project suggested that consultationsmight continue at a rate of 17 to 20 per month. The results of the routineassessment were consistent with trends established during the pilot period.Figure 2: Completed consultations by quarter

n=657

0

10

20

30

40

50

60

70

80

90

Q196

Q2 Q3 Q4 Q197

Q2 Q3 Q4 Q198

Q2 Q3 Q4

M

- - - Moving AverageTrendline

15

Consultations by sitesFigure 3 represents the number of consultations by site over the three years. Twosites, Wetaskiwin and Wainwright, continue to have fewer consultations than theothers, with those at Wetaskiwin decreasing in the last year. Drumheller andDrayton Valley show incremental increases in each year of the evaluation.Although the Edson site continues to have the highest number of consultations,consultations dropped in Edson in 1998-99 by 19%.

The site usage over the three years does not correlate with the community size orpopulation but appears to reflect available psychiatric manpower, distance topsychiatry services and integration with health services. Wetaskiwin, with thelargest population but more days of visiting psychiatry service and the closestsite to Edmonton, continues to have a low referral rate. The Drumheller area,without a visiting psychiatrist, and Drayton Valley, with services only once permonth, have had increases in consultations. Edson, one of the farthest sites frompsychiatric services, had an increase in visiting psychiatric service during theroutine operation, which may explain the decrease in consultations in year three.The two sites, Edson and Drumheller, which have mental health professionals inthe Regional Health Authority acting as site coordinators and liaisons betweenmental health consumers, physicians and mental health clinics, continue to havethe most referrals.Figure 3: Completed consultation by pilot sites April 1, 1996 to March 31, 1999

Total Consults =

13

40

40

7

11

31

84

127

9

12

65

99

102

11

6

0 20 40 60 80 100 120 140

Drayton Valley

Drumheller

Edson

Wainwright

Wetaskiwin

Number of Consultations

1998/991997/981996/97

16

Initial versus repeat consultationsAs would be expected for a new service, the initial six-month period saw themajority of consultations being provided as an initial visit. Examination of theroutine operation in the following two years indicates a ratio of around two newconsultations for each repeat or follow-up consultation (Figure 4). However, 51%of the completed consultations during the three years were for consumers whorequired only one consultation, 59% in the pilot and 50% during routine use(Figure 5).

Figure 4: Percentage of initial and follow-up consultations

N=656

8466 64

1634 36

1996/97 1997/98 1998/99

Initial Follow- up

17

Figure 5: Number of consumers seen only once compared to the totalnumber of completed consults

Figure 6: Percent of consumers who have one versus many visits

83 73 75

17 27 25

Pilot Routine Three Years

Client with one visit only clients with repeat visits

The telepsychiatry service continued as a medical consultation model, asintended, and has not become an ongoing care provider as 75% of consumers areseen only once (Figure 6). Twenty-five percent of consumers had repeat visits,which compares to a 22% repeat rate reported in an Australian evaluation oftelepsychiatry (2). The majority of those who had follow-up consultations hadonly one follow-up, however eight (2%) had five or more visits (Table 4). Theconsumers who were seen more than five times did not significantly differ fromthose who were seen only once in gender, age or diagnosis.

66

271

111

546

0

100

200

300

400

500

600

Pilot Routine

Consultations for Clients seen only once Total Com pleted Consultations

18

Table 4: Number of consultations per client

Type of Consultation Number of Clients

Initial 343 (76%)

Follow Up 66 (15%)

Third 22 (5%)

Fourth 9 (2%)

Five or more 8 (2%)

Total 448

Reason for consultationThe major reason for the consultation was identified by the psychiatrist at thetime of the consultation as the management of a previous diagnosis, with thesecond most common reason for referral being to establish a diagnosis. Familyconsultation, pre-admission screening and post discharge review were otherreasons but were identified for less than 5 consultations overall (Table 5). This isin keeping with the established model of psychiatric consultations.Table 5: Most common reason for consultations

Pilot Routine

Assist with Management 93/111 189/546

Establish Diagnosis 45/111 104/546

Psychiatrist using servicesIn total, during the pilot seven different psychiatrists provided consultationservices, with one individual psychiatrist responsible for 33 consultations, 30% ofall services. Over the next two years, 11 different psychiatrists providedconsultation services, with one providing 40% of the consultations.

Type of service accessedSeventy-five percent of all consultations during the pilot period were for generalpsychiatry, with six of these being for those under 18 years of age. The servicecontinues to serve largely adult psychiatry, with 85% of the completedconsultations for general psychiatry, followed by 10% for geriatric psychiatry.

This result is consistent with the pattern established during the pilot evaluation.At the beginning of the pilot project it was felt by Alberta Hospital Ponoka thatthere would be more of a demand for geriatric services. An examination of theresults over the three-year period did not support this initial supposition.

Referrals by general practitionersThe number of general practitioners who referred consumers for a telepsychiatryconsultation increased incrementally over the three years, with seven out of

19

every ten physicians that referred referring more than one client. Twenty-threepercent of these physicians referred more than ten consumers (Table 6).

The increasing number of physicians using the service and the number of repeatreferrals supports the appropriateness of the clinical recommendations and thesustainability of the service.Table 6: Referrals by general practitioners

Pilot Routine Three Year Summary

Number of Referring Physicians 31 82 N/A

% Referred Once 68 70 70%

% Referred 2-9 N/A N/A 47%

% Referred > 10 N/A N/A 23%

Cancellation of consultationsThe number of canceled consultations increased from 14% to 21% under routineoperation. Most often the client canceled the consultation, either by notificationbefore the appointment or by not showing at an appointment. Not having apsychiatric consultant available was responsible for the cancellation of 1.6% ofthe consultations during both the pilot and routine operation (Table 7).Table 7: Cancellations of consultations

Pilot Routine

Completed 86% 79%

Cancelled 14% 21%

Reasons for Cancellations .

Equipment Failure 0.8 0.4

Operator Failure 0.8 0.1

Consultant Not Available 1.5 1.7

Cancelled by Client 3.1 5.5

Client No show 3.8 4.0

Facility Admission 0.8 0.4

Cancelled by Referring Physician 0 0.4

Cancelled by Local Provider 0.8 0.8

Other 0.8 6.7

Inappropriate Referral 0.8 0.3

Client Changed Mind 0.8 0.4

Client Lacked Transportation 0 0.1

Reason Unknown 0 0.3

20

Consumer characteristicsAgeConsumers during the pilot and routine use (Figure 7) represented all agegroups, remained similar over time and reflect the Alberta age profile (10% over65, 20% 45 to 64, 40% 20 to 44, and 15 % 10 to 19) (Alberta Health PopulationProjections Update April 1999).

The mean age of 40 for routine operation compares to the mean age reported intelepsychiatry projects in Ontario and Ireland (8, 9).Figure 7: Percent of individuals using telepsychiatry by age group

n= 441/447

11% 25% 50% 12%

11% 21% 51% 14%

11% 21% 49% 11%

Age 65 and Older Age 45 to 64 Age 20 to 44 Age 10 to 19

Pilot Routine Three Year

GenderAlmost two-thirds of all consumers using the services are female with thefemales between 20 and 44 the highest users of the service (Table 8). Thiscompares to studies in Australia and in Virginia where women numberedaccounted for 65% and 60% of all their referrals, respectively (2, 6).

Mental health related hospitalizationsSlightly more consumers were hospitalized, in either an acute care or long-termcare facility, at the time of the referral during the pilot operation than in theroutine assessment (Table 8).

21

Table 8: Profile of telepsychiatry consumers pilot versus routine

Pilot Routine

Age Mean

Median

Range

42

40

14-91

40

37

10-95

Gender Male

Female

36%

64%

36%

64%

Mental Health-RelatedHospitalization

Inpatient at the time of the

Consultation

14% 11%

Previous hospitalizations

for mental health problems

41% *

Population at risk Referrals physician indicatedsuicide attempt

13% 10%

Referrals physician

indicated suicide ideation

20% 26%

History of Mental Illness No previous history 7% Year 2*

13%

Previous history >1 year 65%% Year 2*

67%

Family History of MentalIllness

% of referrals that indicatedfamily history

43% 46%

Most Common Reason forReferral

Assist with Management ofCondition

64% 40%

Most Frequent Diagnosis Depression

Major Depression

20%

11%

25%

3%

Physical Health Healthy/No physical Conditions 39% 62%

Most Frequent MedicalDiagnosis: Cardiovascular

15% 6%

* No comparable data as a result of changes in database

Population at riskProviding assistance to those who need timely intervention continues to be afocus of telepsychiatry during routine use, with 10% of consumers having arecent suicide attempt and another 26% having suicide ideation.

22

History of mental illnessFrom the information identified on physician referrals, more clients had a historyof mental illness in year two than in the pilot. The majority of the clientscontinue to be those with a history of mental illness longer than one year. Mostindividuals have seen either their physician or a therapist for some time beforebeing referred to telepsychiatry (Table 8).

Most common reason for referral and provisional diagnosisAs would be expected of a medical consultation model, the most frequent reasonfor referral was “establishing a diagnosis” followed by “ management of acondition”. Depression remained the most frequent diagnosis in both the pilotand the routine and compares with other telepsychiatry studies in Australia andIreland where depression was the most frequent diagnosis of consumers (2, 8).

The occurrence of depression in the population as a whole correlates with theprevalence of depression in the population. One of the consultants suggestedthat usually there are about four or five cases of depression to one case ofschizophrenia in the general population. Dementia occurred as provisionaldiagnosis in 4% of the consumers, both during the pilot and routine operation,suggesting that screening of the geriatric population related to mental illness anddementia would be a continued need (Figure 8).Figure 8: Provisional diagnosis at the time of the consultation

Provisional Diagnosis at Time of ConsultationPercent of Total Consumers n= 324

2011

7655544333322

253

2

44

7

23

22

1

1

11

0

25 20 15 10 5 0 5 10 15 20 25 30

Depression

Anxiety Reaction

Schizophrenia

Personality Disorder

Adjustment Reaction

Psychosis

Substance Abuse

Deferred

Pilot Routine

23

Usual living arrangements and location at the time of the assessmentThe beginning directions for telepsychiatry looked at improving services toindividuals who were in the community. Over 80% of consumers continue to belocated in the community at the time of the assessment with 11 to 14% infacilities at the time of the referral.

The consumer lives in a family situation or with their spouse around 70% of thetime with 12 to 17% living alone (Figure 9).Figure 9: Consumers usual living arrangement at time of initial consult

n=419

43%

27%

17%

10%

4%

44%

30%

12%

9%

3%

2%

1%

0 5 10 15 20 25 30 35 40 45 50

family

spouse/partner

alone

facility

friend

group setting

other

Pilot Routine

Percent

Other consumer characteristicsA profile of consumer information (Figure 10) was identified from the PhysicianReferral forms and the consultation reports. If the physician or consultant didnot mention the particular characteristic for this report, it was assumed that theydid not have that particular characteristic, as these are things that are usuallyasked in a consultation. A structured referral and consultation report have beendesigned to allow for more accurate capturing of this data in the future.Differences between the pilot and routine may be related to physiciandocumentation practices.

0

0

24

Consumers, for the most part, continued to present with proportionately thesame characteristics from the pilot to routine operation. Aggressive behaviourand isolation or withdrawal behaviours are reported much less frequently inroutine operation than in the pilot, while being a receiver of sexual or physicalabuse is mentioned more frequently in the routine operation and less often in thepilot.

Figure 10: Profile of consumers from physician referral form

Assessment of equipmentPerception of sound, picture, use and room/environmentOver 90% of responses during the pilot (n=157) rated the equipment and theroom environment as excellent or good. Lower ratings were given by the pilotparticipants for the quality of sound (73%) and picture (82%). During routineuse, the consumers and consultants were asked a slightly different question, torate the technology as good or needs improvement. Both consumers andconsultants continued to give the equipment positive ratings. Nine-six percent to99% of consumers during routine use rated the equipment as good on thevariables of sound, picture, ease of use and room environment (Figure 11).

Consumer Information Identified from Physician ReferralsPilot and Routine discrete n=447

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Receiver of Sexual Abuse

Receiver of Physcal Abuse

Forensic Involvement

Prescription Drug Abuse

Recreational Drug abuse

SuicideAttempt

Alcohol Abuse

Aggressive Behaviour

Isolation Behaviour

Suicide Ideation

Family History

Pilot Routine

25

As in the pilot, the quality of the sound and of the picture were the areasidentified as needing improvement. However during routine use, fewer of therespondents (8%) suggested improvement was needed in these areas.Figure 11: Assessment of equipment during routine use

96%

4%

92%

8%

97%

3%

93%

7%

99%

1%

95%

5%

99%

1%

99%

1%

Good Needs Improvement Good Needs Improvement

Sound Picture Equipment Use Room Environment

Appropriateness of equipmentConsumers’ perceptions of the appropriateness of video technology for apsychiatric consultation were solicited at the time of the consult. Consumerscontinued to indicate comfort with their ability to interact with the consultantand felt they were able to provide the same information they would have inperson (Table 9).Table 9: Consumer perceptions related to appropriateness of technology

Pilot RoutineI was able to present the same informationas in person

97 % 92%

I felt comfortable with my ability to interactwith the Consultant

98% 91%

Equipment problemsIn the 1998/99 year audio problems were the most often identified problem.Video problems were the second most often recorded problem and includedpixelation and movement delay. Overall, some type of technological problemoccurred in 17% of the consultations during this year. Equipment failure as areason for cancellation was only recorded in 0.1% of the consultations duringroutine operation (Table 10).

During the pilot, equipment problems were not recorded consistently and wereprobably underestimated so no comparison between the pilot and routine usewas available. The pilot evaluation recommended that a more structured

Consumers Consultants

26

tracking of technology problems be considered. Problems during routineoperations were recorded in a log book in the telepsychiatry conference room bythe consultant and site coordinator at the time of use.Table 10: Equipment problems during routine operation

Area # of Problems # of Consults % of Problems

Drayton Valley 17 65 26%

Drumheller 10 99 10%

Edson 14 102 14%

Wainwright 4 11 36%

Wetaskiwin 4 6 67%

Total 49 283 17%

Type of Equipment Problems

Problem Type # of Problems

Video/Audio Synchronization 1

Video 14

Audio 17

Establishing Connection 4

Maintaining Connection 4

Far End Camera Control 9

Although consumers and consultants identified technology problems duringfollow-up phone interviews and in the log books, when asked what impact thishad on the consultation the problem was not considered to have had a negativeimpact. The consultant who completed the majority of the consultationsreported only canceling a consultation once as a result of an equipment problem.In another instance, a consultation continued over the phone when the systemwent down. In both cases, the consultant did not feel the client outcome wasaffected. Consumers did not express concern about equipment problems andappeared to tolerate equipment problems in order to receive a timely service.

AcceptanceConsumersBoth during the pilot and routine use consumers affirmed their satisfaction andacceptance of telepsychiatry during the follow-up telephone interviews. Over90% of the consumers (94% in the pilot and 92% in routine use) agreed that theywould use telepsychiatry again (Table 11). This is comparable to the 88% scorefor this question during a study in South Australia (2). Over 90% of consumers(97% in pilot and 92% during routine use) also indicated that they would

27

recommend telepsychiatry to a friend, which is another indication of satisfactionand acceptance of the technology.Table 11: Consumers perception from telephone interviews

Routine (n=60) and Pilot (n=31)

Yes Definitely or YesPilot Routine

I would use telepsychiatry again 94% 92%

If a friend needed, I would suggest he use telepsychiatry 97% 92%

Referring physiciansReferring physicians also expressed satisfaction with the service with 90%reporting being satisfied or very satisfied overall with the telepsychiatry service.Although physicians were satisfied with the service, several three of the 20commented that face-to-face consultations were preferable. One commented that“telepsychiatry is a reasonable option to a dismal psychiatric service, but what isreally needed is a physical presence”.

Although those consumers who used the service were satisfied withtelepsychiatry service delivery, two physicians suggested that they would havereferred others and didn’t “because the patients did not want to try the service”.

ConsultantsAll psychiatric consultants interviewed during pilot and routine use agreed thattelepsychiatry was an acceptable way to deliver psychiatric consultations.Although they speculated that a follow-up service might also be provided via thetechnology, they did not feel that this was feasible given the manpower availableand the volume of admissions and discharges from Alberta Hospital Ponoka. Inaddition, several considered follow–up to be more appropriate at the communitylevel. The same reservations were also true for the provision of emergencyservices.

The telepsychiatry service paid particular attention to the preparation of thepsychiatrist for the video consultation, including such details as room lightingand making eye contact in their orientation. Although this was appreciated, twopsychiatrists mentioned that it would be unreasonable to believe that they wouldchange long-established therapeutic communication practices.

All psychiatrists commented that the consumers seemed to be comfortable withthe technology.

Psychiatrists were questioned on the applicability of telepsychiatry for geriatricassessment. None were willing to draw any absolute conclusions. It was feltthat many of the geriatric clients required a longer assessment, which would

28

include a medical assessment rather than a psychiatric consultation. It is also ofsignificance that, unlike the adult psychiatry services in the affected RegionalHealth Authorities, the Geriatric Psychiatry service at Alberta Hospital Ponokahas a link through the community mental health services to psychogeriatricoutreach workers. These outreach workers have direct phone access to theGeriatric Psychiatrist, at which time they can complete some initial screening.The outreach worker also mentioned the impact of this direct access in the focusgroup.

Community mental health servicesDuring the initial pilot evaluation a re-occurring theme appeared to be the needfor better integration of the community mental health services with thetelepsychiatry service. During the routine assessment, a focus group was heldwith the mental health therapists from Drayton Valley and Wetaskiwincommunity mental health clinics. The therapist from the Drayton Valley clinicreported that they would now contact the family doctor involved with theirclients to refer to telepsychiatry, if an immediate consultation is necessary.

In addition, therapists noted the need for telepsychiatry for use in emergencysituations; however they were still skeptical of the need for a consultation serviceto the general practitioners apart from the service they provide. ThePsychogeriatric Outreach worker commented that the current practice was to callthe Geriatric Psychiatrist by telephone as the need arose. If further assessmentwas considered to be required, then the individual was usually admitted.

Cost AnalysisThe report of the pilot project (4) included a cost analysis of the telepsychiatryservice, based on a breakeven model. The results indicated that thetelepsychiatry service would become less expensive than services by a travellingpsychiatrist at more than 396 consultations per year.

As noted in a discussion of this result (5), a sensitivity analysis indicated that ifequipment prices were reduced by 10%, the breakeven consultation rate wouldfall to 368 per year. It was also noted that the assumption made that apsychiatrist would see only one patient during a consultation visit tended tofavor telepsychiatry.

A similar cost analysis for the two years of routine operation was undertaken,using the data shown in Table 12. Telepsychiatry is compared with a serviceprovided by a travelling psychiatrist.

29

Table 12: Cost data for psychiatric consultations

Fixed Costs

Total Fixed Costs (TFC) for Telepsychiatry = cost of equipment $456.000

+ cost of installation (depreciated over 4 years) 7,200

+ annual line charge 54,000

TFC =$463,200/4 = $115,800 + $54,000 = $169,800

Variable Costs

Total Variable Costs (TVC) =

Total cost/consult using telepsychiatry = psychiatrist (1 hour) $ 100

+ long distance $ 0.06/min X (75 min) X6 lines 27

+Staff time at sites equipment operation/reception (20 min Eachsite), 40 minutes total X $20/hour

13

TVC Telepsychiatry $ 140

Costs for a Traveling Consult

Total cost/consult using traveling psychiatrist (TVC) =

Psychiatrist (1 hour) $ 100

+travel (average distance to site 200 km X 2 round trip) at$0.30/km

120

+ travel time of psychiatrist at 4 hours X $100 400

+ subsistence 10

TVC Traveling Psychiatrist $ 630

During routine use the cost structure changed slightly, with the long distancerates declining from 37 cents/minute to 6 cents/minute and the cost of travel fora psychiatrist increasing from 25 cents/kilometre to 30 cents/kilometre.

Further data and assumptions were as follows:

! The cost of office space and a receptionist/scheduling service was notincluded in the cost description, as this cost was the same for both servicedelivery models.

! Depreciation over four years was used, as it was in the pilot.

30

! The consultation time was considered to be one hour per consult, plus 15minutes set up time.

! Staff time for equipment set-up and shut-off was considered to be 40 minutesper consult (20 minutes each at receiving and consulting sites). Staff wagewas set at $20/hour.

! Costs were considered on an annual basis.

! Cost of service for a Psychiatrist was the same for both telepsychiatry andtravelling consultation, at $100/hour.

! Travel time to sites was estimated at one hour/100 kilometres.

The number of consults required before the costs of the telepsychiatry serviceand those for services by a travelling psychiatrist would be the same, decreasedto 350 consultations per year from the 396 break even point in the pilot (Figure12). During routine use, the telepsychiatry service averaged 274 consults peryear - below the break-even point.

While actual use of services remained well below the breakeven point, it shouldbe borne in mind that such cost analysis is only one of a number of factors to beconsidered by decision makers. Other points that deserve discussion are theeffect of non-psychiatric use of the network on overall costs and cost of servicesfrom the perspective of the patient.

During the period of routine use, the telehealth network was used frequently foradministrative and clinical committee meetings. This further use of the networkreduces the fixed costs attributable to telepsychiatry.

Information from two sites in one Regional Health Authority indicates that therewere 92 non-telepsychiatry meetings per year over the period of routineoperation. If the total additional meetings per year for the five sites is taken to be150 (as overall use was low at two of them), then there were 424 episodes of useper year, 274 of which were telepsychiatry consultations. Total fixed costattributable to telepsychiatry is then $109,760 ($169,800 x 274/424). On thatbasis, the cost of telepsychiatry consults per year was 274 x $140 + $109,760, or$148,120. That total would have covered only 235 travelling consults so that thenumber of telehealth consultations was well above the breakeven point (224consultations per year).

This estimate is on the same basis as that used for the cost analysis for the pilotproject, with TVC for a travelling consult being based on one consultation pervisit. In many instances more than one consultation would be undertaken. Ifthere were three consultations for each visit by a travelling psychiatrist, the TVCbecomes $277 ($830/3). The breakeven point for the telehealth-travelling consultcomparison is then 801 consultations per year.

31

These different scenarios further illustrate cost issues related to operation of atelepsychiatry service. Additional analysis covering details of the costs andbenefits of the tele-administrative sessions are not considered here.

These calculations are from the perspective of the payer. Further insight isprovided by considering costs from the perspective of the consumer.

In the telephone interviews, consumers indicated that the most important benefitof telepsychiatry was a quicker access time and that they felt telepsychiatry hadhad a positive impact on their condition. Having a timely visit may haveprevented future hospitalizations or mental health professional interactions.Figure 12: Cost comparison travelling consult and telepsychiatry

0

50000

100000

150000

200000

250000

300000

350000

0 50 100 150 200 250 300 350 400 450 500Number of Consultations

Traveling Consult Fixed TP Costs Variable TP Costs Variable & Fixed TP Costs

350

In the follow-up interviews during routine telepsychiatry use, consumers wereasked to describe what the impact would be on them if they had had to drive tothe usual referral center for a psychiatric consultation. Seventy-four percent(23/31) stated that to make a trip to the city they would have to miss time fromwork (45%, 14/31) or pay for child care (29%, 9/31). During the pilot, 25% ofthose interviewed suggested that they might not be able to travel to a largercenter and therefore would not have access to needed mental health treatment.

Cos

ts in

Dol

lars

32

As noted in a discussion on assessment of telehealth applications (7), althoughface to face consults between consumers and a psychiatrist in a larger centrewould always be cheaper than telepsychiatry from the payer's perspective, thecosts borne by society or consumers should not be ignored.

Costs to the patient are made up of travel expenses and the value of lost work (orother activities). With telepsychiatry, assuming that the cost of time lost fromwork is $20/hour and that travel time is 15 minutes, overall cost to the patient isestimated at $30 per consultation. If the consumer has to travel to a major centrefor a consultation, estimated costs from the consumer's perspective rise to $300.This estimate is based on 5.5 hours travel time for 500 km, a one hourconsultation and 0.5 hour wait and registration.

Telepsychiatry offers obvious monetary advantages to that section of theconsumer population who would otherwise have to travel to a major centre. Forthose who have access to a visiting psychiatrist at a centre near their home,telepsychiatry gives advantages through decreasing waiting time, as discussedelsewhere in this report.

Impact on Health Services Delivery

Other services provided by the Alberta mental health servicesOne of the initial and continuing apprehensions voiced by the communitymental health services was the potential impact on the existing mental healthservices. Telepsychiatry might inappropriately be a substitute for the othermental health services.

When asked to comment on the referral patterns after the introduction oftelepsychiatry, 57% of general practitioners suggested that the referral patternremained the same (Figure 13). Several psychiatrists emphasized in interviewsthat follow-up was most appropriately provided in the community. Consumers,general practitioners and psychiatrists all voiced concern about the need foradditional mental health services at the community level. The generalpractitioners commented on the need for more hospital beds. The need foradditional mental health resources at the community level is consistent withinformation collected over the past number of years in Alberta.

33

Figure 13: Referral to mental health clinics

remained the same 57%

more likely to refer17%

less likely to refer13%

no answer13%

The majority of consumers who received a consultation through telepsychiatrywere not previous patients at Alberta Hospital Ponoka or were not admittedafter the telepsychiatry intervention (Table 13). Forty-eight out of the 448discrete consumers seen during the pilot and routine phase were admitted toAlberta Hospital Ponoka. Only 11% of the consumers who used telepsychiatrywere admitted, consistent with the consultation model of the service to supportthe treatment of consumers in the community.

The pattern of Alberta Hospital Ponoka admissions in relation to the timing ofthe telepsychiatry consultation suggests that telepsychiatry for geriatrics wasused more as a screening or pre-assessment process, with most of thoseconsumers requiring admission for a longer assessment. Four percent of theconsumers seen using telepsychiatry had previous admissions to AlbertaHospital Ponoka or had admissions before and after a telepsychiatry consult.More than half of those who were admitted before or after, or both before andafter a telepsychiatry consultation, were also seen multiple times overtelepsychiatry suggesting that these consumers had chronic problems that weredifficult to treat. Telepsychiatry may have prevented more frequent admissionsfor these consumers, but more research is needed to see if this is the case.

A comparison of the follow-up recommendations between the pilot and routine(Figure 14) shows an increase in referrals to local mental health professionals(these would largely be the mental health clinic therapists). In this time periodunique clients served by the clinics continued to increase overall. Telepsychiatrythen becomes another potential source of clients.

34

Figure 14: Follow-up recommendations identified in the consultation report

Table 13: Telepsychiatry consults and admission to Alberta Hospital PonokaAdmission History at Alberta

Hospital Ponoka (AHP*)Number of Telepsychiatry

ConsumersCharacteristics of Consumers

Admitted to AHP# of Consumers not admitted toAHP

400 -

Telepsychiatry Consult beforeadmission to AHP

29 10 were geriatric referrals15 were admitted within 2 weeks of telepsychiatry consult4 had multiple admissions7 had more than 1 telepsychiatry consu

Telepsychiatry Consult afteradmission to AHP

12 2 had multiple admissions withdiagnosis of chronic depression2 consumers had more than 1telepsychiatry consult

Telepsychiatry Consults betweenadmissions to AHP

7 5 had multiple admissions 3 had more than 1 telepsychiatryconsult

Total Telepsychiatry Consumersadmitted to AHP

48 out of 448 discrete patients11%

* Source: AHP inpatient database 1994-1999.

Wait times

Consumers in both the pilot and during routine telepsychiatry use suggestedthat timely appointments were important to them. Thirteen out of 29 physicianrespondents also suggested that getting a consultation sooner was an importantbenefit of telepsychiatry and that immediate service for crisis situations was alsoneeded.

Information about wait times was gathered during the pilot and during routineuse. Wait times are difficult to define and report consistently. Consumerssometimes change or request dates for their convenience (i.e. around holidays,days off work). Psychiatrist availability impacts the number of consultation

2 3 %

4 3 %

1 2 %

1 1 %

1 1 %

3 8 %

2 0 %

2 0 %

7 %

6 %

0 % 5 % 1 0 % 1 5 % 2 0 % 2 5 % 3 0 % 3 5 % 4 0 % 4 5 % 5 0 %

S e e L o c a l M e n t a l H e a l thP r o f e s s io n a l

G e n e ra l P r a c t i t io n e ra n d /o r R x C h a n g e

S c h e d u le dT e le p s y c h ia t r y F o l lo w - u p

A d m is s io n to H o s p i ta l

R e f e r r a l t o A n o t h e rA g e n c y

P i lo t R o u t in e

35

times available at any one time and cancelled and re-scheduled appointmentsaffect calculations. Wait time for use in this study was defined as the timebetween the referral received (appointment made) and the time of the completedconsultation. Cancelled consultations were eliminated from the analysis.

The average wait time over pilot and routine use period was 7.4 days, which wasonly a slight change from the 6.8 averages during the pilot. The frequencies ofwait times are shown in Table 14. Wait times for psychiatric consultations withthe traveling psychiatrist through the mental health clinics or for appointmentsto referral centre psychiatrists were not available. Reports from consumersindicate that physicians tell them that wait times for other psychiatric options aremuch longer, ranging from two weeks to three months.Table 14: Wait time for appointments in telepsychiatry

Time of Wait Between Referral and CompletedAppointment

Number of Consumers

Referral and Appointment same day 52

1-5 days 169

6-10 days 98

11-15 days 35

16 + days 32

Missing observations 61

OutcomesClinical performanceThe referring physicians and the available services in the community influenceclinical improvement following a psychiatric consultation, not only by thequality of the consultation recommendations but also by the support andfollow-up. The consumers’ attitudes, self-esteem and illness behavior, and thecompliance or ability of the consumer to follow through on the recommendedactions are other factors that may affect clinical outcomes and satisfaction (1).Consumers and referring physicians suggested that there were positiveoutcomes during the pilot and routine use as a result of the telepsychiatryconsultation.

Health improvementPhysicians expressed satisfaction with the health improvement of their clientsfollowing the consultation. In the physician survey (n=20), 75% indicated theywere satisfied or very satisfied with the health improvement of their clientfollowing the telepsychiatry consultation, with the remaining reporting beingneutral on this question.

36

Consumers also suggested that clinical outcomes or improvement in their healthhad been positive following the telepsychiatry consults.

The purpose of the Bench Mark Study, which was not completed during thepresent assessment, is to explore the use of quality of life tools in measuringoutcomes following telepsychiatry and face to face consultations. The SF-12 andEuroQol instruments would be used before and after the consultations, both intelepsychiatry and in clinical consultations. The results of this preliminary studywill provide information about the appropriateness of the tools, as well as acomparison of the quality of life and outcomes of consumers between the twoservice delivery models.

SF12-Euroqol pretestThe questionnaires distributed to clients were comprised of two separate toolsincluding the SF12 and the EuroQol. The pretest revealed that many of theclients did not answer the questionnaires to a satisfactory level of completeness.To rectify this problem training sessions were held with the site coordinators todiscuss administrative measures that would help to obtain completeness. Inaddition, the format of the original quality of life questionnaire was modified tomake it easier and less confusing for the client to complete. In general, it hasbeen observed that the combination of the training sessions and themodifications to the questionnaire have improved the level of completeness ofthe survey.

Preliminary Results

From a pool of 147 completed consultations (March 1, 1999 to July 15, 1999) atotal of 88 questionnaires that included the results of the SF12 were received(almost 60%). Of those 88, 53 were deemed complete. Overall, completed formswere received for 36% of the total completed consultations.

The average score for the Physical Component Score was 46 and the averagescore for the SF12 Mental component score was 33. After scoring a sample of 20pretest questionnaires it was found that the Physical Component Score (PCS)was significantly higher than the Mental Component Score (MCS).

SatisfactionService delivery modelPhysicians both during the pilot and the routine phase considered theoperational aspects of the telepsychiatry service to be appropriate and easy touse. Physicians who used telepsychiatry over the pilot and routine period wereasked to rate their satisfaction overall with the telepsychiatry service. Of the 20respondents to the physician survey, 100% reported that they were satisfied orvery satisfied with the referral and scheduling appointment process,communication/consultation notes received from the psychiatrist and the

37

follow-up recommendations (Table 15). One of the physicians was concernedthat it might be difficult to arrange follow-up with the same psychiatrist ifneeded. Satisfaction remained high during routine use.Table 15: Results of physicians survey

N=20 VerySatisfied

Satisfied Neutral Dissatisfied VeryDissatisfied

No Answer

Referral and SchedulingProcess

50 50 0 0 0 0

Communication/ConsultationNotes

75 25 0 0 0 0

Follow-up Recommendations 45 55 0 0 0 0

Health Improvement to theClient

15 60 0 0 0 5

Overall Satisfaction 30 60 10 0 0 0

ConsumersThe overall satisfaction of the consumer, service provider and psychiatrist weregauged by questions asked both at the time of the survey and in the follow-upinterviews.

During the pilot, 84% of consumers and 90% of consultants agreed that they weresatisfied with the session. The level of satisfaction during routine use was 89%.Over the three-year period the ratings were consistently over the target of 80% asset by the Alberta Mental Health Board.

Patients in a study by Barker et al. (1) found that the factor most often linked tosatisfaction with the telepsychiatry service was the quality of the psychiatrist.This appeared to be an influencing factor in the acceptance and satisfaction withthe telepsychiatry service during the study reported here. Although in thetelepsychiatry consultation service delivery model referring physicians were toprovide the consultation results to the consumer, most consumers, duringroutine use, reported that the consultant had given them the most informationand the suggested courses of actions.

Most of those interviewed reported satisfaction with the consultation; howeverthose consumers that reported being somewhat satisfied with the consultationcommented on the consultants’ inability to listen or the consumers’ inability tounderstand the consultant. It was interesting that although the consumers foundthe psychiatrist gave them the most information, most did not remember thename of the consultant.

Referring physicians also expressed satisfaction with the service, with 90%reported being satisfied or very satisfied overall with the telepsychiatry service(Table 15).

38

Discussion

The continued high satisfaction with telepsychiatry service can be attributed to avariety of factors including: reduced wait and travel time, ease of referral andscheduling and the quality of information provided to both client and physician.Consumers reported feeling listened to and supported, which may be as much ameasure of the psychiatrists’ approach style as of the equipment and media use.The qualities of the psychiatrist may prove to be one of the major influences onhow satisfied consumers are with a consultation, and this may not differ betweenface-to-face and telepsychiatry delivery. Other studies have found no significantdifference between the satisfaction rating for televideo and face-to-faceconsultations (3, 9). The work now being completed on a comparison withface-to–face consultations should provide some insight on whether thetechnology actually makes any difference.

Although 29% of those interviewed suggested they would rather usetelepsychiatry than see a psychiatrist in person, most comments received,especially from the family doctors, suggested that face-to-face consultationswould be the more desirable option. The acceptance of telepsychiatry may alsoreflect the limited alternative for timely access to psychiatric consultation in theserural areas. An average wait time for telepsychiatry of 7.4 days was seenadvantageous to both consumers and general practitioners.

Telepsychiatry was introduced as a new service to increase access to psychiatricconsultations. Initially there was some apprehension that it would supplant theexisting service offered. This did not appear to be the case as visits to mentalhealth clinics increased and costs for GP services for individuals with mentalhealth diagnosis maintained or increased in regions with telepsychiatry. Instead,access was improved. In addition, telepsychiatry did not appear to decrease thenumber of psychiatric consultations for the traveling psychiatrists at thecommunity mental health clinics in these areas. Although consumers may beapprehensive about an admission to a psychiatric hospital, not one commentfrom a consumer or doctor referred to any stigma attached to receiving aconsultation from a psychiatric hospital.

Information collected from the psychiatrists follow-up recommendationsindicated an increasing referral rate to local mental health professionals. Theability of community services to fill this demand may be under question.General practitioners in their comments suggested that mental health was stillgreatly under-serviced in their areas. Consumers commented on the need forincreased follow-up in their home community, as did the general practitionersand the psychiatric consultants. Thirty-two percent of those interviewedreported they could not get the mental health service they required when theyneeded it. All of these things appear to substantiate the commonly held

39

perception that there remains considerable unmet mental health needs in ruralAlberta communities.

General practitioners and community mental health therapists interviewedidentified a need for even more timely access to immediate service when anemergency situation arises. Telepsychiatry was not set up to address emergencyneeds. However, the comments form both the pilot and routine use studiessuggest that an under-serviced mental health area may also requiretelepsychiatry on an emergency basis. No attempt was made to determinewhether this need was for consultation only or whether it extended tocertification for admission to hospital.

In the original planning for a telepsychiatry service it was anticipated that thevideoconferencing equipment could be used for a variety of purposes, includingdischarge planning and follow-up. While there was use of the equipment foradministrative purposes, it appeared from the information collected that theseadditional psychiatry-related functions did not occur and further expansionneeds to be carefully considered. Conversations with the psychiatric consultantswould suggest that the expansion of this model to include discharge planningand follow-up would not be desirable because of the limited number ofpsychiatrists, volume of work and the need for service to be provided in thecommunity. In any case, the provision of follow–up service would require amore direct link with community resources, in addition to the generalpractitioners.

It did not appear that the service, with the possible exception of geriatricpsychiatry, was used as a screening process for admission to Alberta HospitalPonoka. Psychiatric consultants suggested that an adequate geriatric assessmentcan often only be made on an inpatient basis where a full medical assessmentand observation can take place.

Several changes in follow-up service recommendations were noted on movingfrom the pilot program to routine service delivery. First, the movement towardsincreased referrals to community health clinics for follow-up suggests thattelepsychiatry service is beginning to integrate or become a part of thecontinuum of services in these communities. The impact of this on communitymental health clinic resources was not explored during the study period.However, it appears that telepsychiatry may increase the demand for localmental health follow- up services. Second, the number of individuals referredfor hospitalization decreased, while the number of individuals referred for repeattelepsychiatry visits increased. Although it is too early to substantiate,telepsychiatry may result in decreased need for hospitalization in someconsumers.

40

Throughout the data collection period the only continuing concerns expressed byproviders and consumers was the need to provide more effective marketingstrategies. More information was needed, both on the services available and thesuccess of the technology in providing consultations. The responsibility for themarketing and information dissemination was shared by the site coordinatorsand the central telepsychiatry service. Despite press releases at the local andprovincial level, virtual lunches with the local physicians and other publicity,several site coordinators mentioned in the final interview that they did notrealize specialized brain injury rehabilitation consultation was available. Inretrospect, perhaps this is an area where closer contact with the existingcommunity agencies, including the mental health clinics, might have beenbeneficial.

The finding that throughout the three years of operation the service did notprovide the necessary 350 consultations to reach the break even point is balanceduse of the video conferencing network for meetings and the significant cost tosome consumers of travel for conventional consults. Alberta, like otherjurisdictions, continues to have a severe shortage of specialists in the rural areas,psychiatrists included. In an unpublished survey of general practitionersundertaken by the Alberta Family Practice Research Network, preliminaryresults indicate that the average wait time for an appointment with a childpsychiatrist in Central Alberta is 11.7 weeks. The average wait time of 7.4 daysin telepsychiatry is of significant advantage.

The impact on the health status of the population is still to be determined. Itwould be the strong recommendation of the evaluators that the service continueto include measures of health status and quality of life to monitor long-termoutcomes. Another area of study may be the impact of the delay of receivingspecialized consultation and the significant burden this places on the consumer,his family and other health services.

Since the completion of the assessment period, the telepsychiatry program hasevolved into a permanent Tele-mental Health service with psychiatrists fromboth Alberta Hospital Edmonton and Claresholm Care Centre offering service.The evaluation of the pilot project (5) played a significant role in obtaining thenecessary credibility and political and community support it required to becomea permanent service. It would be recommended that they continue with themultifaceted evaluation approach, including outcome measures.

PART II

Expansion of Telepsychiatry Services

42

IntroductionIn January 1998, approval was granted to expand the Telepsychiatry Service tosix additional sites, the first of which became operational in July 1998 and the lastin February 1999. The additional sites were at High Level, Fort Vermillion, St.Paul, Mayerthorpe, Rocky Mountain House and Hinton (Table 16).Table 16: Additional sites in the telepsychiatry service

High Level FortVermillion

St. Paul Mayerthorpe RockyMountain

House

Hinton

Population 3093 7980 4861 1669 5805 9961

Distancefrom majorcity (km)

735 650 209 110 220 287

Firstteleconsult

July 1998 August1998

October1998

November1998

December1998

March1999

The success of the telepsychiatry pilot project, and the continuing limited accessto psychiatric services in rural Alberta communities, contributed to theexpansion of the telepsychiatry service. As with the initial site selection,Regional Health Authorities were invited to prepare proposals.

The selection of sites remained somewhat subjective, with the attitude of thecommunity towards innovation, a largely qualitative assessment of mentalhealth needs and the need for the health authorities to strive for equitableresource allocation across the region continuing to be important factors.

The new sites adopted the previously-established psychiatric consultationservice delivery model. Referrals continued to be received from generalpractitioners and consultations provided directly to their patients.

In both the pilot and the expansion project, most of the site coordinators had apsychiatric background in psychiatric nursing, psychology, or mental healththerapy and the majority of rooms are located within a hospital. The onlynotable exception in the service delivery model between the pilot and theexpansion projects is the site in St. Paul, which is located within the psychiatricward of the hospital.

When equipment was purchased for the new sites, a different brand name (VtelLC 5000) was chosen to that used for the pilot sites (CLI Radiance). The newsites also acquired a PC based system that provides more features than the stand-alone system. These features are designed to improve the quality of educationalsessions. It is not anticipated that they will have much of an impact on thequality of telepsychiatry sessions. In addition, based on the recommendations in

43

the pilot project evaluation, a hearing adaptation was also attained; however,there has not yet been an opportunity to use this technology.

Some operational details are outlined in Table 17.

Objectives and methodologyThe expanded sites utilized similar forms and procedures, as did the originalsites. No substantial modification was made to the established database. Sitecoordinators were oriented to the procedures by the operational manager of thetelepsychiatry service. Given the primary focus of the evaluation on the centresthat had participated in the pilot project, the evaluators did not have anopportunity to visit the sites.

44

Table 17: Project description

Fort Vermilion High Level Hinton Mayerthorpe Rocky MountainHouse St. Paul

Room Location Large conferenceroom in mainhospital

Video conferenceroom already inexistence prior totelepsychiatry, inmain hospital

Room located inmain hospital

Large room usedfor education in thebasement of mainhospital

Small visiting roomon emergency unit inmain hospital

Small room locatedon psychiatric unitof main hospital

Site Coordinator Mental HealthTherapist withPsychologyBackground

RegisteredPsychiatric Nurse

Psychologist CharteredPsychologist

RegisteredPsychiatric Nurse

Psychiatric AdminSupport

Marketing Responsibility ofthe sitecoordinator

Responsibility ofthe site coordinator

Responsibility ofthe sitecoordinator

Responsibility ofthe site coordinator

Responsibility of thesite coordinator

Responsibility ofthe site coordinator

Links to MentalHealth Services

Serviceintegrated withmental healthservices in healthcenter andhospital

Service integratedwith mental healthservices

Service isbecomingintegrated withmental healthservices

Service integratedwith mental healthservices in hospitaland surroundingareas

Service integratedwith mental healthservices in mentalhealth clinic andhospital

Service integratedwith mental healthservices in generalhospital

45

Results

UtilizationThe new sites have had a much lower rate of utilization than the pilot sites had inthe initial stages of their establishment. In the time period July 1998 to March1999, there were a total of 37 completed consultations serving 35 discrete clients(Table 18). Rocky Mountain House showed the highest utilization at 13consultations between December 1998 to March 1999. These consultationsaccount for the increase shown in Figure 15.Table 18: Consultations per month by site

Date Site Number persite

TotalConsultations

July, 1998 High Level 2 2Aug, 1998 High Level

Fort Vermilion11

2

Sept, 1998 High Level 1 1Oct, 1998 High Level

Fort VermilionSt. Paul

111

3

Nov, 1998 Fort VermilionMayerthorpe

11

2

Dec, 1998 High LevelFort VermilionRocky Mountain House

233

8

Jan, 1999 High LevelRocky Mountain House

13

4

Feb, 1999 Fort VermilionRocky Mountain House

14

5

March, 1999 High LevelFort VermilionMayerthorpeRocky Mountain HouseHinton

12232

10

Total 37

46

Figure 15: Completed consultations by month

N=37

General psychiatry was the most frequently requested service at 78% of theconsultations, followed by geriatric psychiatry and brain injury, both at 11% ofthe consultations (Figure 16).

Figure 16: Type of service area from July 24, 1998 to March 31, 1999

78%

11%

11%

8

5

1 0

2 21

3

2

4

0

2

4

6

8

1 0

1 2

J u l -9 8 A u g -9 8 S e p -9 8 O c t -9 8 N o v -9 8 D e c -9 8 J a n -9 9 F e b -9 9 M a r -9 9

Num

ber o

f Con

sulta

tions

General Psychiatry Geriatic Psychiatry Brain Injury

N=36

47

Consumer characteristicsThe average age of the population serviced by telepsychiatry is 40, similar to thesites that have been providing routine services. In this short time frame it wouldappear that the new sites are seeing slightly more males. Depression continuesto be the most frequent diagnosis with the new sites for both men and women.However, the proportion who are depressed is higher for women than it is formen (Table 19).Table 19: Consumer profile

Routine Expanded Sites

Age (n=33) MeanRange

4010-95

4011-90

Gender (n=35) Female 64% 49%Most Frequent Diagnosis Depression (25%) Depression

Female n=12 (58%)Male n=14 (43%)

Assessment of equipmentSixty-three percent of clients provided feedback on the quality of the technology.This information was collected from satisfaction questionnaires completed afterthe consultation. Of the clients who responded to the questionnaire, almost allfound the quality of the technology to be excellent. These results are somewhatsurprising, as many of the consultants have observed more technologicaldifficulty and equipment problems with the new sites than was experienced bythe pilot sites.

48

Figure 17: Clients satisfied with the quality of the room, sound, ease of use andpicture

N=22

DiscussionIn discussions with the site coordinators, it was speculated that the low initialutilization might be attributed to the lack of awareness of the service on behalf ofthe general practitioners. This conclusion would be supported by the results,however small, from the survey of physicians.

At the beginning of the expansion of the service it was speculated by some thatthe Rocky Mountain House site would have a slightly different consumer.Specifically, it was felt that the service would see more referrals from males withconcurrent substance abuse and psychiatric diagnosis. It will be interesting tocontinue to monitor the sites to see if this proves to be true.

95%91%

100% 100%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Room Sound Ease of Use Picture

% o

f Clie

nts

Satis

fied

49

References1. Barker DA, Shergill SS, Higginson I, Orrell MW. Patients' views towards

care received from psychiatrists. British Journal of Psychiatry 1996;168:641-46.

2. Clark PH. A referred and patient evaluation of a telepsychiatryconsultation-liaison service in Southern Australia. Journal of Telemedicineand Telecare 1997;3(suppl.1):12-14.

3. Donagier M, Tempier R, lalinec-Michaud M, Meunier D. Telepsychiatry:psychiatric consultation through two-way television. A controlled study.Canadian Journal of Psychiatry 1986;31:32-34.

4. Doze S, Simpson J. Evaluation of a telepsychiatry pilot project. Edmonton:PMHAB/AHFMR, November 1997.

5. Doze S, Simpson J, Hailey D, Jacobs P. Evaluation of a telepsychiatry pilotproject. Journal of Telemedicine and Telecare 1999;5:38-46.

6. Graham M. Telepsychiatry in Appalachia. American Behavioural Scientist1996;39(5):602-615.

7. Hailey D, Jacobs P. Assessment of telehealth applications. Edmonton. AlbertaHeritage Foundation for Medical Research, 1997.

8. Mannion L, Fahy T, Duffy C, Broderick M, Gethins E. Telepsychiatry: anisland pilot project. Journal of Telemedicine and Telecare 1998;4(Suppl. 1):62-3.

9. Stevens A, Doidge N, Goldbloom D, Voore P, Farewell J. Pilot study oftelevideo psychiatric assessments in an underserviced community.American Journal of Psychiatry 1999;156(5):783-85.