Computer-Based Symptom Assessment Is Feasible in Patients With Advanced Cancer: Results From an...

Vol - No - - 2012 Journal of Pain and Symptom Management 1

Original Article

Computer-Based Symptom AssessmentIs Feasible in Patients With Advanced CancerResults From an International MulticenterStudy the EPCRC-CSAMarianne Jensen Hjermstad PhD Hanne C Lie PhD Augusto Caraceni MDDavid C Currow BMed MPH FRACP Robin L Fainsinger MDOdd Erik Gundersen MSc Dagny Faksvaag Haugen MD PhD Ellen Heitzer MDLukas Radbruch MD Patrick C Stone MA MD MRCP Florian Strasser MDStein Kaasa MD PhD and Jon Havard Loge MD PhD on behalf of the EuropeanPalliative Care Research Collaborative (EPCRC)European Palliative Care Research Centre (MJH DFH SK) Department of Cancer Research and

Molecular Medicine Faculty of Medicine Norwegian University of Science and Technology

Trondheim Norway Regional Centre for Excellence in Palliative Care South Eastern Norway

(MJH) Oslo University Hospital Oslo Norway Department of Behavioural Sciences in Medicine

(HCL JHL) Institute of Basic Medical Sciences Faculty of Medicine University of Oslo Oslo

Norway Palliative Care Pain Therapy and Rehabilitation Unit (AC) Fondazione IRCCS Istituto

Nazionale Dei Tumori Milan Italy Discipline Palliative and Supportive Services (DCC) Flinders

University Bedford Park Adelaide South Australia Australia Division of Palliative Care Medicine

Department of Oncology (RLF) University of Alberta Edmonton Alberta Canada Verdande

Technology (OEG) Trondheim and Department of Computer and Information Science (OEG)

Norwegian University of Science and Technology Trondheim Norway Regional Centre of Excellence

for Palliative Care Western Norway (DFH) Haukeland University Hospital Bergen Norway

Division of Oncology (EH) Department of Internal Medicine Medical University of Graz Graz

Austria Department of Palliative Medicine (LR) University Hospital Bonn and Centre of Palliative

Care (LR) Malteser Hospital BonnRhein-Sieg Bonn Germany Division of Population Health

Sciences and Education (PCS) St Georgersquos University of London London United Kingdom

Department of Internal Medicine amp Palliative Care Centre (FS) Cantonal Hospital St Gallen

Switzerland Department of Oncology (SK) St Olavrsquos Hospital Trondheim University Hospital

Trondheim Norway and National Resource Centre for Late Effects After Cancer Treatment (JHL)

Oslo University Hospital Oslo Norway

Abstract

Context Symptom assessment by computers is only effective if it provides valid

results and is perceived as useful for clinical use by the end users patients andhealth care providers

Address correspondence to Marianne Jensen Hjerm-stad PhD Regional Centre for Excellence in Pallia-tive Care Department of Oncology Oslo University

Hospital Ulleval Box 4956 Nydalen Oslo 0424Norway E-mail mjhjermstadmedisinuiono

Accepted for publication October 31 2011

2012 US Cancer Pain Relief CommitteePublished by Elsevier Inc All rights reserved

0885-3924$ - see front matterdoi101016jjpainsymman201110025

2 Vol - No - - 2012Hjermstad et al

Objectives To identify factors associated with discontinuation timeexpenditure and patient preferences of the computerized symptom assessmentused in an international multicenter data collection project the EuropeanPalliative Care Research Collaborative-Computerized Symptom Assessment

Methods Cancer patients with incurable metastatic or locally advanced diseasewere recruited from 17 centers in eight countries providing 1017 records foranalyses Observer-based registrations and patient-reported measures on paindepression and physical function were entered on touch screen laptopcomputers

Results The entire assessment was completed by 949 (nfrac14 965) with medianage 63 years (range 18e91 years) and median Karnofsky Performance Status(KPS) score of 70 (range 20e100) Predictive factors for noncompletion werehigher age lower KPS and more pain (P 0012) Time expenditure amongcompleters increased with higher age male gender Norwegian nationalitynumber of comorbidities and lower physical functioning (P 0007) but wasinversely related to pain levels and tiredness (P 003) Need for assistance waspredicted by higher age nationality other than Norwegian lower KPS and lowereducation (Plt 0001) More than 50 of patients preferred computerizedassessment to paper and pencil version

Conclusion The high completion rate shows that symptom assessment bycomputers is feasible in patients with advanced cancer However reducedperformance status reduces compliance and increases the need for assistanceFuture work should aim at identifying the minimum set of valid screeningquestions and refine the software to optimize symptom assessment and reducerespondent burden in frail patients J Pain Symptom Manage 2012--e- 2012 US Cancer Pain Relief Committee Published by Elsevier Inc All rights reserved

Key Words

Advanced cancer symptom assessment computer technology patient-reported outcomes datacollection

IntroductionPatients with advanced cancer experience

multiple symptoms with fluctuating intensityand severity during the disease trajectory12

Unsystematic symptom assessment or the useof tools that are not adapted to a particularpopulation or do not possess the necessary psy-chometric properties may lead to suboptimaltreatment and inadequate symptom relief34

The use of electronic devices for symptom as-sessment offers several benefits compared withthe traditional paper and pencil method Theseinclude advantages with respect to data collec-tion and storage instant calculation and imme-diate presentation of scores data quality andtracking of symptoms over time Rapid transferof results alsomay facilitate follow-upof patientsoutside hospitals Moreover computerized as-sessment may be targeted to the individualif programmed to automatically select or skip

questions based on the patientrsquos previous an-swers thereby reducing the respondent bur-den Such processes also may allow tailoredinformation tobeprovided to patients and theircaregivers Computerized versions of frequentlyused assessment tools for patient-reported out-comes (PROs) such as the Short Form (36)Health Survey5 have been developed for usewith different digital devices and platformsamong others by the Quality Metric group6

and the Patient-Reported Outcomes Measure-ment Information System7 and are forthcom-ing for the European Organization forResearch and Treatment of Cancer Quality ofLife Questionnaire-Core 308e10

Usability is a frequently used term in relationto computerized assessment of PROs It refers toa set of concepts in the International Organiza-tion for Standardization standards with differ-ent definitions depending on whether the

Vol - No - - 2012 3Computerized Symptom Assessment in Advanced Cancer

standards focus on the process or the product11

One problem related to commercially devel-oped software applications is the lack of de-tailed knowledge about the end users forexample patients and health care providersSymptom assessment by computers is only effec-tive if it provides valid results is perceived as use-ful and is the preferred assessment method bypatients and health care providers In our opin-ion important aspects of usability in patientswith advanced disease include execution timeuser satisfaction and an interface that facilitatesdata entry without assistance Usability maybe evaluated by compliance andor numberof missing items Previous research from ourgroup has shown that higher age and poor per-formance status predicted the need for helpwhen using a pilot version of a computerizedsymptom assessment (CSA) tool12

The European Palliative Care ResearchCollaborative (EPCRC)13 recently closed aninternational multicenter study in patientswith advanced cancer using computers for self-report of symptoms and other PROs A majorstudy objective was to further develop a com-puter-based assessment tool for pain assessmentand classification based on our previous ver-sion12 with the aim of producing a useful toolfor clinical work and research14

The primary aim of the present report was toexamine the feasibility of the EPCRC-CSA toolby identifying factors associated with comple-tion Specific research questions focused ondifferences between completers and noncomp-leters in relation to disease extent age gendereducational level performance status cognitivefunction and nationality The number of itemscompleted before discontinuation was countedto see if there was a potential threshold levelfor response burden Time expenditure andneed for assistance also were examined andpatient preferences were assessed

MethodsData collection was performed using touch-

sensitive computers (HP Compaq TC4200L

1200 tablet PCs Hewlett-Packard DevelopmentCompany LP Palo Alto CA) The software wasdeveloped using JAVA and Eclipse for touchscreen computers running Windows XP2 Reg-istration consisted of two parts one to be filledin by health care professionals and the other

by patients These could be completed in ran-dom order Questions that were not takenfrom existing validated questionnaires in thefour languages used (English German Italianand Norwegian) were subject to detailed for-ward and backward translations according tothe established procedures15

All data were entered by tapping directly onthe screen with a stylus and the time expendi-ture for completion of each item was automati-cally registered One or two questions weredisplayed in bold font on the screen at any onetime A screenshot displaying an example ofthe layout is presented in the Appendix Eachitem had to be answered before proceeding tothe next If the patient chose to abandon dataentry all data up to that point were stored TheEnglish German Italian and Norwegian ver-sions were all programmed with a similar layout

Software ContentThe health personnel part of the assessment

consisted of a record of sociodemographicand medical variables the Mini-Mental StateExamination (MMSE) for assessment of cogni-tive function16 Karnofsky Performance Status(KPS)17 two items related to primarysecondarycachexia and the Edmonton Classification Sys-tem for Cancer Pain18

The patient section comprised one set offorms to be answered by all patients the Edmon-ton Symptom Assessment System (ESAS) (nineitems)19 the European Organization for Re-search and Treatment of Cancer Quality of LifeQuestionnaire-Core 30 (30 items)20 one screen-ing item about pain intensity (PI) at its worst inthe last 24 hours taken from the Brief PainInventory21 15 items on physical function22

the Brief Patient Health Questionnaire(PRIME-MD PHQ9) for depression (nine ques-tions 10 items)23 twoquestions from the Subjec-tive Global Assessment of Nutritional Intake(SGA)24 (six items) plus five questions (sevenitems) related to need for assistance patientsrsquoopinions on time expenditure and preferencesfor computerized vs paper- and pencil-based as-sessment A computerized pain body map wasalso included Data on this is not reported

Different paths were automatically selected bythe computerbasedon thepatientsrsquo responses tothe PI screening question PathA with a PI scorelt1 therewereno furtherpainquestions andpa-tients were automatically routed to the items on


physical functioning depression nutrition anduser satisfactionPathBwithaPI score$1 therewas one section on general PI neuropathicpain (painDetect)25 and pain etiology and onescreening question on breakthrough pain(BTP) leading to one of the two further auto-matically selected paths Path B1 if BTP wasindicatedby the screeningquestion the comput-erized version of the Alberta Breakthrough PainAssessment Tool (Alberta BTP Tool)26 was ad-ministered If BTP was not identified by thescreening question patients were routed toPath B2 this consisted of a shorter modified ver-sion of the Alberta BTP Tool assessing generalpain relief After completion of the relevantquestions patients following Paths B1 or B2were routed to the remaining items on physicalfunctioning depression nutrition and user sat-isfaction as inPathAPatientswho confirmedde-creased food intake on the SGA were given oneadditional questiononnutritional intake beforeproceeding to the final items

The software version of the Alberta BTPTool and the shortened modified version andthe PHQ9 plus the questions from SGA hadto be programmed in with more items thanin the original paper versions to allow for mul-tiple response options This made the toolslonger Thus the maximum number of itemsrequiring a response differed across pathsPath A 79 Path B1 167 and Path B2 143Fig 1 displays the full questionnaire matrixand number of items in the different paths

Sampling and Data CollectionInclusion criteria were cancer patients with

incurable metastatic or locally advanced dis-ease those aged 18 years or older and whowere able and willing to provide written in-formed consent The exclusion criterion wasinability to take part in a symptom assessmentbecause of physical or obvious cognitive impair-ment or language problems

The EPCRC project started in Norway inOctober 2008 with the UK Austria GermanySwitzerland Italy Canada and Australia consec-utively entering the study yielding 17 participat-ing centers in all Patients were included frompalliative care inpatient and outpatient unitshospices and general oncology and medicalwards Patients were asked about their willing-ness toparticipate by a physician study coordina-tor or research nurse who were also available

during the assessment for provision of help ifnecessary Encrypted backups were made onUSB sticks after each assessment Every twoweeks the memory sticks were sent by airmailto the trial office in Trondheim Norway fordata extraction

PatientsThestudywas closed inDecember2009A total

of 1070 registrations were extracted from theUSB sticks Nineteen of these were excluded be-cause of withdrawal of informed consent(nfrac14 4) or because no data were recorded owingto technical failure (nfrac14 15) leaving a total of1051 registrations Another 34 patients who hadconsented to participate were excluded fromthe analyses because the file contained no(nfrac14 24)oronlyonecompleted itemfromthepa-tient questionnaire (nfrac14 4) or because the timeof completion exceeded 24 hours as a result ofincorrect procedures when saving data (nfrac14 6)These 34 patients were significantly older

mean age 67 vs 62 years (t1043frac14197Pfrac14 0049) but did not differ on performancestatus (704 vs 700) educational attainmentor survival time compared with the 1017 patientswho were included in further analyses

Ethical ConsiderationsThe study was performed according to the

rules of the Helsinki Declaration An identifica-tion number was automatically assigned to eachpatient on the first data entry No patient-identifiable information could be retrievedfrom theUSB sticks that were used for data stor-age Ethical approval was obtained before thestart of the study at each site All participantsprovided written informed consent

Statistical AnalysesPotential differences between patients who

completed and those who did not completethe assessment were investigated using Pear-sonrsquos chi-squared test and independent samplet-tests for categorical and continuous variablesrespectivelyLogistic regression models were used to ex-

amine predictors for aborting or completingthe assessment and identify factors related tothe need for assistance As a result of a relativelysmall number of noncompleters the number ofvariables that could be entered into the multi-ple model was limited

Fig 1 Overview of the full questionnaire matrix with number of completers (nfrac14 965) and noncompleters(n frac14 52) The figure displays the order of questions as presented to the patients After the PI screening itemthere were different paths automatically selected by the computer based on the PI score Path A with a PI scorelt1 there were no further pain questions and patients were automatically routed to the items on physical func-tioning Path B with a PI score $1 there was one section on general pain neuropathic pain (painDetect) andpain etiology and one screening question on BTP leading to one of the two automatically selected paths Path B1if BTP was confirmed the computerized version of the Alberta BTP tool was administered Path B2 with no BTPpatients were given a shorter modified version of the Alberta BTP tool on general pain relief Paths B1 and B2were then routed to the items on physical functioning Patients who confirmed weight loss on the SGA were givenone extra item on nutritional intake before proceeding to the final items Maximum number of items are as fol-lows Path A 79 Path B1 167 and Path B2 143 1Percentage of noncompleters 2scored on a 0e10 NRS 3totalnumber of items in the computerized version used differed from the number in the traditional paper versions toallow for multiple response options PIfrac14 pain intensity BTPfrac14 breakthrough pain SGAfrac14 Subjective GlobalAssessment of Nutritional Intake NRSfrac14 numerical rating scale ESASfrac14 Edmonton Symptom Assessment SystemEORTC QLQ-C30frac14 European Organization for Research and Treatment of Cancer Quality of Life QuestionnaireCore 30 Alberta BTP toolfrac14 Alberta Breakthrough Pain Assessment Tool PHQ9frac14 Brief Patient HealthQuestionnaire



Multiple linear regression analyses were usedto investigate the factors predictive of the timetaken to complete the assessment Because thenumber of responses (completed items) per pa-tient variedbetween74and167 theaverage timespent per response per participant was used asa measure of time expenditure (dividing totaltime spent to complete the assessment by thenumber of responses made) Time expenditurewas positively skewed and was therefore logtransformed to achieve normality Time expen-diture was then regressed on a range of illnessand demographic variables in the initial modelage groups gender country (not Norway vsNorway) education (three categories lt1010e12 and gt12 yearsuniversity) spousenospouse survival (three categories 1e9091e270 and 271thorn days) use of opioids (yesno) KPS (divided by 10) European Organiza-tion for Research and Treatment of Cancer(EORTC) physical function (divided by 10)the nine ESAS items whether the patient re-quired assistance or not and the MMSE catego-rized as no (27e30) potential (24e26) ordefinite cognitive deficit (10e23) according tothe previously applied cutoffs in advanced can-cer patients2728 Nonsignificant predictors weresequentially deleted from the model

For all regression analyses multivariate out-liers were screened for using the following crite-ria leverage values higher than three times theaverage leverage value andorhaving a standard-ized residual lower or higher than three SDsfrom the mean Nonsignificant predictors weresequentially deleteduntil only significant predic-tors remained in themodels A P-value of 005 orless was considered statistically significant Bon-ferroni corrections for multiple comparisonswere not applied because all analyses were ex-plorative29 The PASW 18 statistical package wasused with all analyses (SPSS Inc Chicago IL)

ResultsThe final study sample comprised 1017 pa-

tients The majority of those who were included(nfrac14 965 949) completed the entire assess-ment (Fig 1)

Noncompleters vs CompletersFifty-two patients aborted the assessment

at different time points (referred to as

noncompleters) No significant differenceswere found between completers and non-completers with respect to gender distributioneducational attainment living situation lan-guage group number of medical comorbid-ities current disease status or treatmentsetting (inpatients vs outpatients) (Table 1)However thereweremorepatientswithpoten-

tial ordefinitecognitivedeficit amongnoncomp-leters than completers (c2frac14 3383 Plt 0001)Significantlymorenoncompleters livedashorterlength of time after completing the assessmentrelative to completers (Pfrac14 0010) and weremore likely to receive opioid therapy (P frac140004) (Table 1) They also were older (meanage 66 years [SD 123] vs 62 years [SD 124]Pfrac14 0017) and had lower performance status(mean KPS score 53 [SD 191] vs 71 [SD 163]Pfrac14 0001) data not tabulatedNoncompleters reported significantly lower

mean scores on the EORTC physical functionscale (Pfrac14 0001) and scored significantly high-er on the following four ESAS scales paindrowsiness tiredness and appetite indicatinghigher symptom intensity (Pfrac14 0019e0039)(Table 2)Overall amedian of 50 items (range 10e139)

was answered by the noncompleters beforeaborting the assessment Only two of the non-completers were automatically subject to theskip session onpain having a PI scorelt1 as dis-played in Fig 1 Seven of the noncompleterswho were routed to the pain section did nothave BTP and were consequently not presentedwith the Alberta BTP Tool but were exposed tothe general pain relief questions Fig 2 displaysthe actual number of items answered by the 52noncompleters before aborting the assessmentAlthough therewasnoobvious cutoff indicatinga potential threshold for respondent burdenthe most marked decline was found between40 and 50 items (Fig 2) This was after the initialitems from the ESAS and the EORTC and dur-ing the general pain section The percentageof noncompleters at each step also is displayedin Fig 1 Noncompliance also was examinedin relation to KPS score with significantly feweritems completed in patients with a KPS of 40 orbelow (data not shown) Themost frequent rea-sons for noncompletion were reported as lsquolsquotootiredrsquorsquo (nfrac14 19) lsquolsquoother reasons not specifiedrsquorsquo(nfrac14 11) and lsquolsquoassessment being too longrsquorsquo(nfrac14 10)

Table 1Patient Characteristics

VariablesCompleters

(nfrac14 965) n () MissingNoncompleters(nfrac14 52) n () Missing c2 P

GenderMale 502 (520) 0 22 (423) 0 186 0172Female 463 (480) 30 (577)

Education (years)lt10 333 (346) 3 22 (431) 1 299 022510e12 336 (349) 19 (373)gt12 293 (305) 10 (196)

Living situationSpouse 631 (655) 2 33 (635) 0 093 0761No spouse 332 (344) 19 (365)

LanguageNorwegian 485 (503) 0 22 (423) 0 380 0283Germana 208 (216) 16 (308)Englishb 173 (179) 11 (214)Italian 99 (103) 3 (58)

MMSE sum scoresc

27e30 755 (801) 22 33 (688) 4 966 000824e26 113 (120) 5 (104)10e23 75 (80) 10 (208)

Treatment settingInpatients 543 (564) 2 35 (648) 0 24 0121Outpatients 420 (435) 19 (352)

Survival days from assessment1e90 240 (259) 38 21 (438) 4 926 001091e270 252 (272) 6 (125)271thorn 435 (469) 21 (438)

Comorbidity (numbers)0 478 (496) 2 19 (365) 0 647 00911 322 (334) 23 (423)2 124 (129) 6 (115)3 or more 39 (40) 5 (96)

Primary cancer diagnosisd

Digestive organs 254 (264) 3 9 (173) 0Breast cancer 167 (174) 5 (96)Respiratory organs 161 (167) 11 (212)Male genital organs 104 (108) 6 (115)Urinary 55 (57) 4 (77)Leukemia or lymphoma 43 (45) 3 (58)Skin cancermalignant melanoma 40 (42) 0 (0)Malignant connectivesoft-tissue tumors 31 (32) 3 (58)Head 28 (29) 3 (58)Gynecological 26 (27) 4 (77)Secondaryill-defined malignant tumors 24 (25) 2 (38)Otherd 29 (30) 2 (38)

Current disease statusMetastatic 815 (846) 2 46 (885) 0 056 0453Loco-regionally advanced 148 (154) 6 (115)

BTP in last 24 hoursNo 384 (571) 292e 7 (135) 33 307 0080Yes 289 (429) 12 (231)

Ongoing treatment with opioidsNo 420 (435) 0 12 (211) 0 844 0004Yes 545 (565) 40 (769)

MMSEfrac14Mini-Mental State Examination BTPfrac14 breakthrough painBold values indicate statistical significanceaIncludes England Australia and CanadabIncludes Germany Austria and SwitzerlandcMMSE mean scores 27e30frac14 no cognitive dysfunction 24e26frac14 potential cognitive dysfunction 10e23frac14 definite cognitive dysfunction27dIncluding tumors of the central nervous system (15 and zero) malignant endocrine tumors (seven and two) multiple primary cancers (four andzero) and malignant bone tumors (three and zero)eQuestions on BTP were only presented to patients with a pain intensity score of gt1 (Fig 1)


Table 2Mean Scores Physical Functioning (EORTC) and Symptoms (ESAS) Completers vs Noncompleters

Completers (nfrac14 965) Noncompleters (nfrac14 52)

PMean SD Mean SD t

EORTC physical functioninga 5578 2689 3493 2511 536 lt0001

ESASPainb 216 238 296 247 236 0019Tirednessb 369 254 444 263 208 0037Nauseab 112 199 152 239 133 NSDepressionb 191 234 196 244 015 NSAnxietyb 206 233 223 259 051 NSDrowsinessb 338 249 419 265 229 0022Appetiteb 327 302 415 289 207 0039Feeling of well-beingb 344 237 385 264 119 NSShortness of breathb 192 240 223 275 091 NS

EORTCfrac14 European Organization for Research and Treatment of Cancer ESASfrac14 Edmonton Symptom Assessment System NSfrac14 nonsignificantaOn a scale of 0e100 higher score implies better functioningbOn a scale of 0e10 higher score implies higher symptom intensity


More than 50 of the completers were Nor-wegians Compared with patients from allother countries combined Norwegian patientshad higher levels of education (c2frac14 443Plt 0001) and lived significantly longer afterthe assessment (c2frac14 1997 Plt 0001) Therewere no differences with respect to age gen-der MMSE scores or opioid therapy whencomparing Norwegians with participants fromother countries

Predictors for CompletionThe variables significantly correlated (Spear-

manrsquos rho) with noncompletion (nfrac14 1002excluding 15 patients missing variables) wereopioid treatment (Pfrac14 0012) comorbidities

Fig 2 Overview of number of completed items before abortscreening question routing patients to Paths B1 or B2 if PI$for routing patients to the full or shorter modified version o53 PIfrac14 pain intensity BTPfrac14 breakthrough pain

(Pfrac14 0018) KPS score (Plt 0001) physicalfunctioning (Plt 0001) education (Pfrac14 0026)and ESAS pain (Pfrac14 0009) tiredness (P frac140038) drowsiness (Pfrac14 0009) and appetite(Pfrac14 0012)These variables were included as predictors in

the initial logistic regression model Nineteenmultivariate outliers were then removed leaving983 cases Table 3 displays the final model Non-completion compared with completion was pre-dicted by higher age lower KPS score andhigher pain levels (Pfrac14 0001e0029)

Predictors for Need for AssistanceMore than half of the sample (555) specifi-

cally responded that they did not require any

ing the assessment (noncompleters nfrac14 52) 1The PI1 was question no 40 2The BTP screening questionf the Alberta BTP Assessment Tool was question no

Table 3Predictors for Noncompletion (nfrac14 983) and Need for Assistance (nfrac14 955)

B SE Exp (B) 95 CI P

Noncompletiona

Age groupb 040 0182 149 1041e2124 0029KPS group 117 0160 031 0227e0426 lt0001ESAS pain 021 0077 123 1060e1433 0007Constant 009 1155 110 0936

Need for assistancec

Age group 042 0063 152 1345e1723 lt0001Countryd 057 0153 176 1308e2380 lt0001KPSe 034 0050 071 0644e0784 lt0001Educationf (years) lt0001lt10 147 0197 436 2963e6417 lt000110e12 093 0193 254 1743e3708 lt0001

Constant 108 0548 034 0050

KPSfrac14 Karnofsky Performance Status ESASfrac14 Edmonton Symptom Assessment SystemaCompletionfrac14 reference category (1) noncompletionfrac14 predicted category (0)bAge groups 18e27 28e37 38e47 48e57 58e67 68e77 78e87 and 88thorn yearscNo need for assistancefrac14 reference category (1) need for assistancefrac14 predicted categorydCountry (dichotomized other vs Norway)eKPS is calculated for each 10-point increasefUniversity-level education was the reference category


assistance in completing the assessment The var-iables significantly correlated with need vs noneed for assistance (Spearmanrsquos rho) were re-tained for inclusion in the final model basedon 955 patients after removing two outliers age(Plt 0001) country (Plt 0001) opioid treat-ment (Pfrac14 0001) comorbidities (Plt 0001)KPS (Plt 0001) physical functioning (P lt0001) survival (Plt 0001) education (P lt0001) MMSE (Plt 0001) and ESAS tiredness(Pfrac14 0038) nausea (Pfrac14 0002) and feeling ofwell-being (Pfrac14 0001) Higher age countryother than Norway lower KPS score and lowereducational level (Plt 0001) were the signifi-cant predictors of need for assistance (Table 3)

Time ExpenditurePatients used an average of 1608 (95 CI

1513e1703) 2910 (95 CI 2718e3102)and 2446 (95 CI 2332e2600) minutes tocomplete Path A (79 responses) Path B1(167 responses) and Path B2 (143 responses)respectively The mean time taken was 112 sec-onds per response (95 CI 434e2240)

The initial multiple linear regression modelis described in the statistical analyses sectionand predicts time expenditure expressed as av-erage time in seconds used per response (logtransformed) Patients with missing values forany of the variables were excluded from theanalysis leaving 907 cases Six multivariate out-liers were further removed

Greater time expenditure was significantlyassociated with male gender higher ageNorwegian nationality number of comorbid-ities lower self-reported physical functioningless pain and less tiredness on the ESAS (allP 003) The final model explained 127of the variance in time expenditure per re-sponse (R2 adjusted) (Table 4)

Females spent on average 010 seconds lessper response thanmales whereasNorwegian pa-tients spent an average of 012 seconds per itemmore than patients from the other countriesAverage time per response increased with 004seconds per 10 year increase in age 005 secondsper 10 decrease in physical functioning and005 seconds for each additional comorbidity

To explore whether time per response in-creased with a higher number of completeditems the time taken (log transformed) to com-plete the PHQ923 for patients following Path A(nfrac14 290) was compared with that in patientsin Path B1 and Path B2 (nfrac14 669) after exclu-sion of six outliers Patients in Path A used1402 seconds more to complete the PHQ9compared with those who also had completedthepain items (mean17745 and16343 respec-tively Plt 0001 effect size Cohenrsquos dfrac14 029)

Patient PreferencesOf the 965 patients who completed the full

assessment 32 reported having no previousexperience with computers 203 had little

Table 4Factors Predicting Higher Time Expenditure Per Response (nfrac14 901)a

B SE Seconds per item (95 CIs) t P

Gender (male)b 005 0013 010 (0147 to 0045) 355 lt0001Age groupc 002 0005 004 (0014 to 0062) 317 0002Countryd 005 0013 012 (0057 to 0186) 384 lt0001Comorbidities 002 0008 005 (0014 to 0089) 269 0007Physical functioninge 002 0003 005 (0056 to 0034) 748 lt0001

ESASPain 001 0003 002 (0032 to 0007) 298 0003Tiredness 001 0003 001 (0027 to 000) 213 0034

Constant 110 0042 2634 lt0001

The final model explained 127 of the variance in time expenditure per response (R2 adjusted)aAverage time in seconds per responsedlog transformed multiple linear regressionbMale gender is the reference categorycAge groups 18e27 28e37 38e47 48e57 58e67 68e77 78e87 and 88thorn yearsdCountry (dichotomized other vs Norway)ePhysical functioning is calculated for each 10-point increase


236 had some and 237 said they had a lotof previous experience Four hundred ninety-eight patients (516) responded that theywould have preferred computer assessment topaper and pencil if given a choice whereasone-third expressed no preference for one orthe other The vast majority (832) regardedthe time taken to complete the assessment asacceptable

DiscussionThe overall impression from this interna-

tional study with more than 1000 respondentsis that symptom assessment by computers is fea-sible in patients with advanced cancer regard-less of place of care culture and languageThis is in line with smaller single-center studiesin different cancer samples81230e35

Our study yielded a very high compliancerate with 95 of the patients completing thefull assessment Furthermore more than 50of the patients did not require any assistancewhen filling in the lengthy assessment whichrequired a minimum of 79 and a maximumof 167 responses (depending on the initial PIscore) In our opinion this is promising withrespect to the development of future versionsof the computerized tool

Nosignificantdifferenceswere foundbetweenthe 965 completers and 52 noncompleters withrespect to gender distribution educational at-tainment living situation language group num-ber of comorbidities current disease status ortreatment setting (inpatient vs outpatient) as

shown in Table 1 This was somewhat surprisingbecause these variables normally exert a pro-found influence on the overall health conditionand thereby the ability to participate in clinicalstudies Furthermorepatientswithahigher levelof functioning are most likely to receive treat-ment as outpatients Thus it could be assumedthat place of care may be an important variablein relation to study completion However thevast differences in the organization and deliveryof palliative care at the different sites both na-tionally and internationally make treatment set-ting an unreliable variable particularly so ina cross-sectional study with a convenience sam-ple From a clinical point of view however thepredictive factors for noncompletion (higherage low performance status and more pain)are well-known indicators of current disease sta-tus and show that the frailest patients are gener-ally unable to complete assessment tools likethis12 regardless of treatment setting The factthat 28 of the 34 patients who initially had con-sented to participate did not complete any orjust one of the self-report items reflects the gen-eral problemof low study compliance among thesickest patients3637 However these patientsdefinitely need targeted symptomatic treatmentand the use of tailored questionnaires withscreening items for the most prevalent symp-toms supplemented by information from signif-icant others may be an optionThe current assessment was comprehensive

with a minimum number of 79 items repre-senting a considerable respondent burden inpatients with advanced disease The reasonsfor this rather extensive questionnaire were


that the present study represented one step inthe development of a computer-based assess-ment tool and that the software included spe-cific questionnaires for collecting data onpain depression and cachexia respectivelyas described in the goals for the EPCRC13 Itcould be argued that this sample witha mean KPS score of 70 may not be representa-tive of most patients with advanced cancerHowever performance status and physicalfunction are shown to vary considerably in pa-tients with advanced cancer in general1438

Our sample also was characterized by a highsymptom burden with more than 50 receiv-ing opioid treatment Forty-three percent ofthose reporting pain had had BTP in the last24 hours before the assessment and 85 hadmetastases indicating advanced disease

Assistance and Time ExpenditureIt was interesting that more than 50 of the

patients did not need any assistance when us-ing the computers Although informationabout the need for assistance with computer-ized tools is scarce12 this finding is most likelyrelated to the relatively high performance sta-tus of the patients but also may indicate thatCSA was perceived as useful by the patientsNorwegian patients were least likely to need as-sistance with completing the instrument 37vs 52 in the other countries combined Rel-evant predictors such as performance statuswere controlled for in the multivariate analy-ses However it may be speculated that Norwe-gian patients nevertheless had a higher level offunctioning that enabled the completion ofa lengthy questionnaire This was because sig-nificantly more Norwegians than patientsfrom other countries were recruited from gen-eral oncology wards rather than palliative careunits However this may only express organiza-tional policies in cancer care both at the soci-etal and hospital levels and can only be fullyexplored with another study

It also could be that there was a somewhatdifferent approach to performing the assess-ment across countries The study coordinatorsin countries other than Norway might havebeen more prone to offer assistance when in-forming patients about the study or Norwe-gian patients may have expressed a desire toperform the assessment on their own This in-terpretation is supported by the finding that

the time expenditure among the Norwegianparticipants was significantly longer Unfortu-nately such differences are difficult to elimi-nate in a multicenter study like the presentone

The mean time taken to complete the threepaths was 16minutes (79 responses) 29minutes(167 responses) and 24 minutes (143 re-sponses) respectively numbers that correspondreasonablywell to the average timeof 15minutes(60 responses) reported in a previous study fromour group12 Mean time per item was 1116 sec-onds in the present study whereas Fyllingenet al12 reported a mean time expenditure of15e17 seconds per response A study by Velikovaet al35 compared paper and computer versionsof two frequently used questionnaires (44 items)and reported an average of 11 seconds per itemin the computer-based version Interestinglyhowever slightly shorter time expenditure wasfound when using computers relative to thepaperandpencil format in their studyThe stron-gest predictors of time expenditure in the pres-ent study were country gender self-reportedphysical functioning and number of medicalcomorbidities the latter two emphasizing theneed to offer assistance if necessary especiallyin the frail patients14

Themean time expenditure per item was notsignificantly influenced by a higher number ofitems We found an average difference of only14 seconds for completing the last question-naire in the assessment the PHQ9 between pa-tients who completed the shortest path (PathA)and those patients who completed the longerpaths (Paths B1 and B2) Although we werenot able to identify any threshold level for re-spondent burden in number of items it is plau-sible that a higher number of items may lead tomore dropouts Despite the very low dropoutrate overall 31 (60) of the noncompletershad aborted the questionnaire before questionno 53 It was surprising that time expenditurewas inversely related to pain levels and tired-ness a finding that could be investigated in laterstudies Although it is difficult to fully explainthis a possible explanation may be that pain isa dominating symptom and that patients whoare in pain and are tired tend to spend lesstime reflecting on each item to get the assess-ment over and done with If so this also maybe related to the finding that patients followingthe shortest path Path A (no pain items) took


more time completing the PHQ9 than patientsin Paths B1 and B2

Patient PreferencesThe vast majority of patients regarded the

time required to complete the assessment aslsquolsquoabout rightrsquorsquo This was a comprehensive as-sessment used for academic purposes and weare well aware that a completion time of 29minutes is not compatible with routine assess-ment of symptoms in clinical practice How-ever it is an important finding that patientsindicated a clear preference for using com-puters compared with paper and pencil whenspecifically asked in line with results fromother studies1231343539 This is particularly in-teresting given the fact that most patients hadhad little experience with computers

StrengthsSeveral strengths of this study are worth point-

ing out First and foremost the sample size (withmore than1000 respondents) and thehighnum-ber of participating centers make this one of thelargest international studies in patients with ad-vanced cancer to date and lend confidenceabout the generalizability of the dataMany stud-ies on computerized assessments and usability is-sues incancer includingpatientpreferences andtime expenditure have been carried out in sin-gle institutions with relatively small numbers ofparticipants although some studies have sam-ples exceeding 100 patients81239

The use of validated questionnaires is impor-tant in any collection of PROs regardless ofthe assessment method Even if not all thequestionnaires used in the present study hadbeen subject to specific validation for elec-tronic capture of data we have reason to be-lieve that their performance on the computeryielded valid results for an explorative studylike this one This is supported by the findingsfrom a meta-analysis with 233 direct compari-sons of paper and pencil vs computerizedforms in 46 studies using both applicationsshowing only minor differences40 The stan-dardized translation processes that were usedalso are likely to reduce possible biasing factorsrelated to culture and language

The highest compliance rate was found inNorway This could be attributed to the factthat 93 of Norwegian households have com-puters at home and 79 use the Internet every

day representing the highest numbers in Eu-rope according to the Eurobarometer studies41

On the other hand about 50 of the patientsin our study had had no or little experiencewith computers This may indicate that light-weight devices and a user-friendly interfacewith few items on the screen at a time largebold fonts an appealing layout and distinct ar-rows clearly indicating how to proceed may out-weigh the importance of prior computerexperience This is an important finding with re-gard to usability and is further supported by thefact that most patients preferred computerizedassessments over paper formsMissing data are a particular challenge for

studies in patients with advanced disease Ifthe pattern of missing items is not randomas is often the case in this patient group thismay represent bias37 One advantage with com-puterized assessments is the elimination ofmissing responses because the software canbe programmed to require that every item ischecked before proceeding to the next aswas done in the present study We considerthe completion rate (95) in our study to behighly acceptable There were however signif-icant differences between the characteristics ofcompleters and noncompleters Counting andcharacterizing noncompleters is thereforeimportant in any computerized assessment todetect a potential response bias

LimitationsThe main limitation of the present study is re-

lated to the lack of information about patientswho were either not informed about the studyor who declined participation Ethical regula-tions in most countries do not permit registra-tion of any medical or sociodemographiccharacteristics of nonparticipants We cannotrule out that patients in the poorest conditionswere not included because of reasons such asgatekeeping and barriers on the part of healthcare providers as pointed out previously12 Thismay skew the results in thedirectionofa lsquolsquohealthysample effectrsquorsquo thereby overestimating the posi-tive results related to overall compliance thelow number of nonresponders and user accept-ability Elderly people who donot use computerson a regular basis may be generally skepticalabout modern technology and may thereforerefuse to participate in computer studies To tryto reduce the possible bias of including mostly


computer-savvy patients the informed consentform deliberately did not present the study asa computer investigation but as an assessmentof frequent cancer symptoms that should be reg-istered on a laptop computer This way of intro-ducing the study also was emphasized in theinstructions to the study coordinators Althoughwe cannot rule out that the study was labeled asa computer study in the oral information givenat some of the sites the fact that 50 of the pa-tients had little experience with computerswould not support this It also may be arguedthat the length of the assessment was a limitingfactor requiring up to 167 responses Howeverit is important to remember that the assessmenttool used in the present study was not the finalversion of a computerized assessment tool butconsisted of items selected for specific researchpurposes in the EPCRC

Most patients preferred an electronic for-mat However a substantial number of patientspreferred the paper and pencil format butcompleted the data collection on the com-puter nevertheless Despite the fact that thiswas not a comparative study we believe thatthe high completion rate and the relativelyshort average time expenditure strongly indi-cate that computerized assessment is highlyfeasible in this group of patients Our clinicalimpression is that the practical obstacles facinga bedridden elderly patient who is asked to re-spond to a multipage paper-based question-naire are substantial and possibly bigger thanthose presented by responding on a computer

Modern computer technology offers severaladvantages over paper forms such as immediatepresentation of results compilation of aggregatescores and sophisticated skip sessions that mayreduce the number of irrelevant items Advancesin software solutions and use of appropriatehardware such as different lightweight tabletcomputers iPads or cellphonesmay in fact in-crease the response rate in frail patients whomay be discouraged when presented with longcumbersome paper forms All these factors inprinciple should facilitate the clinical uptakeof this method for symptom assessment To suc-ceed however it is of the utmost importance toacknowledge and address the barriers against in-troduction and use of modern technology forsymptom assessment Reports have shown thatsuch barriers exist among health care providersand support staff4243 but to a smaller extent

among patients43 A key factor for successful im-plementation is the active involvement in allphases of the implementation process It is im-portant that the new assessment methods areperceived by the users as quick user-friendlyand clinically relevant with aminimumof techni-cal problems Furthermore results from elec-tronic data capturing must be integrated intomedical records to be useful Unfortunatelythis integration is problematic in many hospitalsettings today because of confidentiality or in-compatibility issues

We believe however that symptom assess-ment regardless of format can only have a trueimpact on clinical practice when applied system-atically Monitoring may then facilitate doctor-patient communication4445 and identify areasinneedof clinical attention leading to improvedsymptom control Whether symptom assessmentshould be performed by computers or on paperin the last stages ofdisease remains anopenques-tion that needs to be examined

In our opinion future research in this areashould continue in two parallel directions to re-duce respondent burden The first is to definerelevant and valid screening questions with suffi-cient psychometric properties to disclose symp-toms that need more thorough assessment andtargeted interventions The other is to continuethe development of software technology by im-proving themethodology for computer adaptivetesting46 and the user interface aiming at appli-cations compatible with all sorts of frequentlyused hardware devices Todayrsquos rapid advancesin computer science offer ample opportunitiesfor collecting PROs undoubtedly representingthe future for symptom assessment

ConclusionThis study confirmed that electronic assess-

ment of symptoms is well accepted by patientswith advanced cancer and that a user-friendly in-terface seems to outweigh the lack of computerexperience Assistance may be needed andshould be available for elderly and frail patientswith reduced performance status Future workshould focus on the identification of aminimumnumber of screening questions about the mostprevalent symptoms to reduce respondent bur-den and on the continued development of thesoftware content and technology


Disclosures and AcknowledgmentsThe authors express their gratitude to study

coordinatorresearch nurse Gunnhild Jakobsenand research nurse Torbjoslashrn Oslashvreness at theEuropean Palliative Care Research Centre fortheir efficient administrative work Specialthanks also are given to the study coordinatorsat the participating centers and the patientswho took part in the study

The EPCRC (2006e2010) was funded by theEuropean Commissionrsquos Sixth Framework Pro-gramme (contract no LSHC-CT-2006-037777)with the overall aim to improve treatment ofpain depression and fatigue through transla-tion research Core scientific groupwork pack-age leaders S K (project coordinator) FrankSkorpen M J H and J H L Norwegian Uni-versity of Science and Technology GeoffreyHanks University of Bristol A C and FrancoDe Conno Fondazione IRCCS Istituto Nazio-nale dei Tumori Milan Irene Higginson KingrsquosCollege London F S Cantonal Hospital StGallen L R RWTH Aachen UniversityKenneth Fearon University of EdinburghHellmut Samonigg Medical University of GrazKetil Boslash Trollhetta AS Norway Irene Rech-Weichselbraun Bender MedSystems GmbHAustria O E G Verdande Technology ASNorway Scientific advisory group Neil Aaron-son The Netherlands Cancer Institute VickieBaracos and R L F University of Alberta P CS St Georgersquos University of London MariLloyd-Williams University of Liverpool Projectmanagement S K Ola Dale and D F HNorwegian University of Science andTechnology

There are no financial benefits or conflictsof interest that might bias this work

References1 Johnsen AT Petersen MA Pedersen LGroenvold M Symptoms and problems in a nation-ally representative sample of advanced cancerpatients Palliat Med 200923491e501

2 Laugsand EA Kaasa S de Conno F Hanks GKlepstad P Intensity and treatment of symptomsin 3030 palliative care patients a cross-sectional sur-vey of the EAPC Research Network J Opioid Manag2009511e21

3 Fadul NA El Osta B Dalal S Poulter VABruera E Comparison of symptom burden amongpatients referred to palliative care with hematologic

malignancies versus those with solid tumors J PalliatMed 200811422e427

4 Teunissen SC Wesker W Kruitwagen C et alSymptom prevalence in patients with incurable can-cer a systematic review J Pain Symptom Manage20073494e104

5 Ware JE SF 36 health survey manual and inter-pretation guide 1st ed Boston MA New EnglandMedical Center 1993

6 Quality Metric Incorporated Generic healthsurveys Available from httpwww qualitymetriccomWhatWeDoGenericHealthSurveystabid184Default aspx Accessed July 11 2011

7 PROMIS Patient-Reported Outcomes Measure-ment Information System Available from httpwwwnihpromisorg Accessed July 11 2011

8 Erharter A Giesinger J Kemmler G et al Imple-mentation of computer-based quality-of-life monitor-ing in brain tumor outpatients in routine clinicalpractice J Pain Symptom Manage 201039219e229

9 Petersen MA Groenvold M Aaronson NK et alDevelopment of computerized adaptive testing(CAT) for the EORTC QLQ-C30 physical function-ing dimension Qual Life Res 201120479e490

10 CHES Computer-based Health Evaluation Sys-tem Available from httpwwwchesat AccessedJuly 18 2011

11 Abran A Khelifi A Suryn W Seffah A Usabilitymeanings and interpretations in ISO standardsSoftw Qual J 200311325e338

12 Fyllingen EH Oldervoll LM Loge JH et alComputer-based assessment of symptoms and mo-bility in palliative care feasibility and challengesJ Pain Symptom Manage 200938827e836

13 EPCRC European Palliative Care ResearchCollaborative Available from httpwwwepcrcorgAccessed June 23 2011

14 Kaasa S Loge JH Fayers P et al Symptom as-sessment in palliative care a need for internationalcollaboration J Clin Oncol 2008263867e3873

15 EORTC European Organisation for Researchand Treatment of Cancer Available from httpwwweortc be Accessed July 1 2011

16 Folstein MF Folstein SE McHugh PR Mini-Mental State a practical method for grading thecognitive state of patients for the clinicianJ Psychiatr Res 197512189e198

17 Karnofsky D The use of nitrogen mustards inthe palliative treatment of carcinoma with particularreference to bronchogenic carcinoma Cancer 19481634e656

18 Fainsinger RL Nekolaichuk CL A lsquolsquoTNMrsquorsquo clas-sification system for cancer pain the EdmontonClassification System for Cancer Pain (ECS-CP)Support Care Cancer 200816547e555


19 Bruera E Kuehn N Miller MJ Selmser PMacmillan K The Edmonton Symptom AssessmentSystem (ESAS) a simple method for the assessmentof palliative care patients J Palliat Care 199176e9

20 Aaronson NK Ahmedzai S Bergman B et alThe European Organisation for Research and Treat-ment of Cancer QLQ-C30 a quality-of-life instru-ment for use in international clinical trials inoncology J Natl Cancer Inst 199385365e376

21 Daut RL Cleeland CS Flanery RC Develop-ment of the Wisconsin Brief Pain Questionnaire toassess pain in cancer and other diseases Pain198317197e210

22 Helbostad JL Oldervoll LM Fayers PM et al De-velopment of a computer-administeredmobility ques-tionnaire Support Care Cancer 201119745e755

23 Spitzer RL Kroenke K Williams JB Validationand utility of a self-report version of PRIME-MDthe PHQ primary care study Primary Care Evalua-tion of Mental Disorders Patient Health Question-naire JAMA 19992821737e1744

24 Detsky AS McLaughlin JR Baker JP et al Whatis subjective global assessment of nutritional statusNutr Hosp 200823400e407

25 Freynhagen R Baron R Gockel U Tolle TRpainDETECT a new screening questionnaire toidentify neuropathic components in patients withback pain Curr Med Res Opin 2006221911e1920

26 Hagen NA Stiles C Nekolaichuk C et al TheAlberta Breakthrough Pain Assessment Tool for can-cer patients a validation study using a delphi pro-cess and patient think-aloud interviews J PainSymptom Manage 200835136e152

27 Kurita GP Sjogren P Ekholm O et al Preva-lence and predictors of cognitive dysfunction inopioid-treated patients with cancer a multinationalstudy J Clin Oncol 2011291297e1303

28 Lawlor PG Gagnon B Mancini IL et al Occur-rence causes and outcome of delirium in patientswith advanced cancer a prospective study ArchIntern Med 2000160786e794

29 Nakagawa S A farewell to Bonferroni the prob-lems of low statistical power and publication biasBehav Ecol 2004151044e1045

30 Kearney N Kidd LMiller M et al Utilising hand-held computers to monitor and support patientsreceiving chemotherapy results of aUK-based feasibil-ity study Support Care Cancer 200614742e752

31 Carpenter JS Rawl S Porter J et al Oncologyoutpatient and provider responses to a computerizedsymptom assessment system Oncol Nurs Forum200835661e669

32 Abernethy AP Herndon JE Wheeler JL et alFeasibility and acceptability to patients of a longitu-dinal system for evaluating cancer-related symptomsand quality of life pilot study of an eTablet data-collection system in academic oncology J PainSymptom Manage 2009371027e1038

33 Abernethy AP Herndon JE Wheeler JL et alImproving health care efficiency and quality usingtablet personal computers to collect research-quality patient-reported data Health Serv Res2008431975e1991

34 Mullen KH Berry DL Zierler BK Computer-ized symptom and quality-of-life assessment for pa-tients with cancer part II acceptability andusability Oncol Nurs Forum 200431E84eE89

35 Velikova G Wright EP Smith AB et al Auto-mated collection of quality-of-life data a comparisonof paper and computer touch-screen question-naires J Clin Oncol 199917998e1007

36 Jordhoy MS Kaasa S Fayers P et al Challengesin palliative care research recruitment attritionand compliance experience from a randomizedcontrolled trial Palliat Med 199913299e310

37 Fielding S Fayers PM Loge JH Jordhoy MSKaasa S Methods for handling missing data in palli-ative care research Palliat Med 200620791e798

38 Borgsteede SD Deliens L Francke AL et alDefining the patient population one of the prob-lems for palliative care research Palliat Med 20062063e68

39 Fortner B Okon T Schwartzberg L Tauer KHouts AC The Cancer Care Monitor psychometriccontent evaluation and pilot testing of a computeradministered system for symptom screening andquality of life in adult cancer patients J Pain Symp-tom Manage 2003261077e1092

40 Gwaltney CJ Shields AL Shiffman S Equivalenceof electronic and paper-and-pencil administration ofpatient-reported outcome measures a meta-analyticreview Value Health 200811322e333

41 Special Eurobarometer 340 lsquolsquoScience andTechnologyrsquorsquo Report No 340Wave 3701 BrusselsBelgium European Commission 2010

42 Andre B Ringdal GI Loge JH Rannestad TKaasa S Implementation of computerized technol-ogy in a palliative care unit Palliat Support Care2009757e63

43 Clark K Bardwell WA Arsenault T DeTeresa RLoscalzo M Implementing touch-screen technologyto enhance recognition of distress Psychooncology200918822e830

44 Velikova G Booth L Smith AB et al Measuringquality of life in routine oncology practice improvescommunication and patient well-being a random-ized controlled trial J Clin Oncol 200422714e724

45 Berry DL Blumenstein BA Halpenny B et alEnhancing patient-provider communication withthe electronic self-report assessment for cancera randomized trial J Clin Oncol 2011291029e1035

46 Wainer H Computerized adaptive testingA primer 2nd ed Mahwah NJ Lawrence ErlbaumAssociates 2000


Appendix

Screenshot showing one of the pain items in the computerized tool


Objectives To identify factors associated with discontinuation timeexpenditure and patient preferences of the computerized symptom assessmentused in an international multicenter data collection project the EuropeanPalliative Care Research Collaborative-Computerized Symptom Assessment

Methods Cancer patients with incurable metastatic or locally advanced diseasewere recruited from 17 centers in eight countries providing 1017 records foranalyses Observer-based registrations and patient-reported measures on paindepression and physical function were entered on touch screen laptopcomputers

Results The entire assessment was completed by 949 (nfrac14 965) with medianage 63 years (range 18e91 years) and median Karnofsky Performance Status(KPS) score of 70 (range 20e100) Predictive factors for noncompletion werehigher age lower KPS and more pain (P 0012) Time expenditure amongcompleters increased with higher age male gender Norwegian nationalitynumber of comorbidities and lower physical functioning (P 0007) but wasinversely related to pain levels and tiredness (P 003) Need for assistance waspredicted by higher age nationality other than Norwegian lower KPS and lowereducation (Plt 0001) More than 50 of patients preferred computerizedassessment to paper and pencil version

Conclusion The high completion rate shows that symptom assessment bycomputers is feasible in patients with advanced cancer However reducedperformance status reduces compliance and increases the need for assistanceFuture work should aim at identifying the minimum set of valid screeningquestions and refine the software to optimize symptom assessment and reducerespondent burden in frail patients J Pain Symptom Manage 2012--e- 2012 US Cancer Pain Relief Committee Published by Elsevier Inc All rights reserved

Key Words

Advanced cancer symptom assessment computer technology patient-reported outcomes datacollection

IntroductionPatients with advanced cancer experience

multiple symptoms with fluctuating intensityand severity during the disease trajectory12

Unsystematic symptom assessment or the useof tools that are not adapted to a particularpopulation or do not possess the necessary psy-chometric properties may lead to suboptimaltreatment and inadequate symptom relief34

The use of electronic devices for symptom as-sessment offers several benefits compared withthe traditional paper and pencil method Theseinclude advantages with respect to data collec-tion and storage instant calculation and imme-diate presentation of scores data quality andtracking of symptoms over time Rapid transferof results alsomay facilitate follow-upof patientsoutside hospitals Moreover computerized as-sessment may be targeted to the individualif programmed to automatically select or skip

questions based on the patientrsquos previous an-swers thereby reducing the respondent bur-den Such processes also may allow tailoredinformation tobeprovided to patients and theircaregivers Computerized versions of frequentlyused assessment tools for patient-reported out-comes (PROs) such as the Short Form (36)Health Survey5 have been developed for usewith different digital devices and platformsamong others by the Quality Metric group6

and the Patient-Reported Outcomes Measure-ment Information System7 and are forthcom-ing for the European Organization forResearch and Treatment of Cancer Quality ofLife Questionnaire-Core 308e10

Usability is a frequently used term in relationto computerized assessment of PROs It refers toa set of concepts in the International Organiza-tion for Standardization standards with differ-ent definitions depending on whether the













































VariablesCompleters






MMSE sum scoresc

27e30 755 (801) 22 33 (688) 4 966 000824e26 113 (120) 5 (104)10e23 75 (80) 10 (208)













PMean SD Mean SD t
















Noncompletiona




Constant 108 0548 034 0050
















Constant 110 0042 2634 lt0001































































































Appendix














































VariablesCompleters






MMSE sum scoresc

27e30 755 (801) 22 33 (688) 4 966 000824e26 113 (120) 5 (104)10e23 75 (80) 10 (208)













PMean SD Mean SD t
















Noncompletiona




Constant 108 0548 034 0050
















Constant 110 0042 2634 lt0001































































































Appendix




PMean SD Mean SD t
















Noncompletiona




Constant 108 0548 034 0050
















Constant 110 0042 2634 lt0001































































































Appendix




Noncompletiona




Constant 108 0548 034 0050
















Constant 110 0042 2634 lt0001































































































Appendix






Constant 110 0042 2634 lt0001































































































Appendix























































































Appendix


Computer-Based Symptom Assessment Is Feasible in Patients With Advanced Cancer: Results From an...

Documents

Transcript of Computer-Based Symptom Assessment Is Feasible in Patients With Advanced Cancer: Results From an...