The Clinical Effectiveness of Web-Based Cognitive Behavioral Therapy With Face-to-Face Therapist...

Original Paper

The Clinical Effectiveness of Web-Based Cognitive BehavioralTherapy With Face-to-Face Therapist Support for DepressedPrimary Care Patients: Randomized Controlled Trial

Ragnhild Sørensen Høifødt1, PsyD; Kjersti R Lillevoll1, PsyD; Kathleen M Griffiths2, PhD; Tom Wilsgaard3, PhD;

Martin Eisemann1, PhD; Knut Waterloo1, PhD; Nils Kolstrup4, MD, PhD1Department of Psychology, Faculty of Health Sciences, University of Tromsø, Norway2Centre for Mental Health Research, The Australian National University, Canberra, Australia3Department of Community Medicine, Faculty of Health Sciences, University of Tromsø, Norway4Department of Community Medicine, General Practice Research Unit, Faculty of Health Sciences, University of Tromsø, Norway

Corresponding Author:Ragnhild Sørensen Høifødt, PsyDDepartment of PsychologyFaculty of Health SciencesUniversity of TromsøTromsø, 9037NorwayPhone: 47 776 49230Fax: 47 776 45291Email: [email protected]

Abstract

Background: Most patients with mild to moderate depression receive treatment in primary care, but despite guidelinerecommendations, structured psychological interventions are infrequently delivered. Research supports the effectiveness ofInternet-based treatment for depression; however, few trials have studied the effect of the MoodGYM program plus therapistsupport. The use of such interventions could improve the delivery of treatment in primary care.

Objective: To evaluate the effectiveness and acceptability of a guided Web-based intervention for mild to moderate depression,which could be suitable for implementation in general practice.

Methods: Participants (N=106) aged between 18 and 65 years were recruited from primary care and randomly allocated to atreatment condition comprising 6 weeks of therapist-assisted Web-based cognitive behavioral therapy (CBT), or to a 6-weekdelayed treatment condition. The intervention included the Norwegian version of the MoodGYM program, brief face-to-facesupport from a psychologist, and reminder emails. The primary outcome measure, depression symptoms, was measured by theBeck Depression Inventory-II (BDI-II). Secondary outcome measures included the Beck Anxiety Inventory (BAI), the HospitalAnxiety and Depression Scale (HADS), the Satisfaction with Life Scale (SWLS), and the EuroQol Group 5-Dimension Self-ReportQuestionnaire (EQ-5D). All outcomes were based on self-report and were assessed at baseline, postintervention, and at 6-monthfollow-up.

Results: Postintervention measures were completed by 37 (71%) and 47 (87%) of the 52 participants in the intervention and54 participants in the delayed treatment group, respectively. Linear mixed-models analyses revealed significant difference in timetrends between the groups for the BDI-II, (P=.002), for HADS depression and anxiety subscales (P<.001 and P=.001, respectively),and for the SWLS (P<.001). No differential group effects were found for the BAI and the EQ-5D. In comparison to the controlgroup, significantly more participants in the intervention group experienced recovery from depression as measured by the BDI-II.Of the 52 participants in the treatment program, 31 (60%) adhered to the program, and overall treatment satisfaction was high.The reduction of depression and anxiety symptoms was largely maintained at 6-month follow-up, and positive gains in lifesatisfaction were partly maintained.

Conclusions: The intervention combining MoodGYM and brief therapist support can be an effective treatment of depressionin a sample of primary care patients. The intervention alleviates depressive symptoms and has a significant positive effect onanxiety symptoms and satisfaction with life. Moderate rates of nonadherence and predominately positive evaluations of the

J Med Internet Res 2013 | vol. 15 | iss. 7 | e153 | p.1http://www.jmir.org/2013/7/e153/(page number not for citation purposes)

Høifødt et alJOURNAL OF MEDICAL INTERNET RESEARCH

XSL•FORenderX

http://www.jmir.org/article/downloadSuppFile/2714/9663

http://www.w3.org/Style/XSL

http://www.renderx.com/

treatment also indicate the acceptability of the intervention. The intervention could potentially be used in a stepped-care approach,but remains to be tested in regular primary health care.

Trial Registration: Australian New Zealand Clinical Trials Registry number: ACTRN12610000257066;https://www.anzctr.org.au/trial_view.aspx?id=335255 (Archived by WebCite at http://www.webcitation.org/6GR48iZH4)

(J Med Internet Res 2013;15(7):e153) doi:10.2196/jmir.2714

KEYWORDS

cognitive therapy; therapy, computer-assisted; Internet; mental health; depression; randomized controlled trial; primary healthcare

Introduction

OverviewDepression is a highly prevalent disorder that often causessubstantial functional impairment in daily life, reduction inquality of life, and increased medical service utilization [1-6].There exist several effective psychological and pharmacologicaltreatments for depression [7]. However, a large proportion ofthose suffering from this disorder receive inadequate treatmentor no treatment at all [8,9]. Cognitive behavioral therapy (CBT)has proven to be as effective as pharmacotherapy in treatingmild to moderate depression, with the benefit of reduced ratesof relapse [10,11].

Internet-Based Treatment of DepressionThe principles and techniques of CBT have been extensivelydisseminated through self-help books and computer- orInternet-based programs. A substantive body of research showsthat Internet-based CBT can be an efficacious treatment ofdepression (eg, [12-15]). Research also suggests that suchinterventions are cost-effective compared to face-to-facetreatments because they result in symptom reduction andreduced burden of disease for patients and alleviate demandson clinician time and resources [16-18].

Self-help can be self-administered or guided by a therapist,although the active involvement of the therapist in guidedself-help is less extensive than in conventional psychologicaltherapy. Studies generally show small to moderate effects ofself-administered, unguided CBT in the treatment of depression[19-23], although in some studies unguided interventions haveyielded large treatment effects [24,25]. Still, an increasingamount of research has pointed to the importance of support inInternet-based interventions, with interventions offering somedegree of support from a professional during treatment generallyshowing substantially larger treatment effects than interventionsinvolving little or no professional support [26-28]. However,this conclusion is primarily based on meta-analytic results; theresults from the few studies directly comparing guided andunguided interventions are mixed [14,24,29]. Overall, guidedinterventions show moderate to large treatment effects fordepression, and the average effect sizes for guided self-help arecomparable to the effects of time-limited face-to-face treatment(eg, [13-15,30]). This is further supported by a recentmeta-analysis, which found no significant differences in effectbetween guided self-help and face-to-face therapy [31].

MoodGYM is a free Web-based CBT program developed toprevent and treat mild to moderate depression [32]. Studies havedemonstrated the effectiveness of MoodGYM in reducingsymptoms of depression and anxiety among public registrants,trial participants, callers to a national helpline service, users ofthe UK National Health Service portal, adolescent school-basedpopulations, and in Australian and Norwegian student samples[20,24,25,29,33-35]. Positive effects have been shown to besustained over 12 months [36]. However, few previous trialshave investigated the effect of MoodGYM combined withtherapist support. A study found that the conjunction ofMoodGYM and face-to-face therapy was superior to bothMoodGYM alone and for some outcome measures, totime-limited face-to-face therapy alone [29]. Also the resultsof a cluster-randomized trial suggested positive effects of thecombination of MoodGYM and support from generalpractitioners (GPs) compared to GP care alone [37].

Depression Treatment in Primary Health CareMost patients with psychological problems will receive mostor all of their mental health care in primary care, and findingssuggest that many patients prefer to consult their GP fortreatment of depression [2,38-40]. Clinical practice guidelinesprimarily recommend treating mild to moderate depressionusing psychosocial interventions [41,42], and this is also inaccordance with reported patient preferences [43-45].Nevertheless, structured psychological interventions areinfrequently delivered in general practice [46-48] because oftime constraints [49-51] and a lack of knowledge andcompetence among GPs in the delivery of evidence-basedpsychological interventions [51,52]. The use of CBT-basedself-help resources could be a way to improve the delivery ofpsychological interventions in general practice. This wouldallow for short consultations and for the clinician to be afacilitator rather than a cognitive therapist. These features couldimprove feasibility in general practice, where the volume ofpatients is high and it is essential that interventions are briefand practical.

Aim of the StudyThe current study was designed to trial a procedure fordepression treatment that could be suitable for implementationin general practice. The project was planned as the first phaseof research for this treatment, with the second phase focusingon further evaluation carried out in everyday general practice.The aim was to evaluate the effectiveness and acceptability ofa guided self-help intervention combining the MoodGYMprogram with brief face-to-face therapist support in a sample



XSL•FORenderX




of primary care patients with mild to moderate symptoms ofdepression. This was investigated in a randomized controlledtrial comparing the guided self-help intervention to adelayed-treatment control condition. The primary hypothesiswas that therapist-supported Web-based CBT would lead to alarger reduction in depressive symptoms than the controlcondition. To determine if the intervention was acceptable topatients, satisfaction with treatment, adherence, and reasons fordropout were investigated.

Methods

Study DesignThe study was a randomized controlled trial with balancedrandomization (1:1). Participants were randomly allocated to atreatment condition comprising 6 weeks of Web-based CBTwith therapist support, or to a 6-week waitlist for the sametreatment during which time they could also access treatmentas usual. The study was conducted at the Department ofPsychology at the University of Tromsø where a small self-helpoutpatient clinic was established. The research protocol wasapproved by the Regional Committee for Research Ethics inNorthern Norway (2011/2163) and the Human Ethics Committeeof the Australian National University (ANU). The trial wasregistered in the Australian New Zealand Clinical Registry(ACTRN12610000257066). The trial is reported in accordancewith the CONSORT-EHEALTH [53] (see MultimediaAppendices 1-3).

ParticipantsParticipants (N=106) were recruited between October 2010 andOctober 2012 from GPs, primary care nurses, and from waitlistsof primary care referrals at 2 psychiatric outpatient clinics.Calculations of required sample size were based on a power of.80, significance level of .05 (2-sided), and an expected effectsize of 0.6 on depressive symptoms at posttest. The estimationsnecessitated a sample size of 45 participants per group. Amedian dropout rate between 17% and 19% has been reportedfor computerized or Web-based treatment programs [54,55].With a 20% expected dropout, a total sample size of 108 wasrequired to gain sufficient power, yielding group sizes of 54participants.

Local GPs and primary care nurses were informed about thestudy both verbally at practice meetings and through writteninformation. They provided patients who they considered asmildly to moderately depressed based on clinical appraisaland/or screening instruments with written information aboutthe project. Potentially eligible patients on waitlists forpsychiatric outpatient treatment were identified by clinic staffand subsequently received information by postal mail from theresearch group. When informing a patient of the project, allrecruiters were asked to send a notification to the research groupby using a prepaid envelope. The notification simply notifiedthe researchers that a patient had been informed of the projectand did not reveal any information about the patient. Patientswere provided with general information about the treatment andthe aim of the project and detailed information about themethods for handling issues of privacy and anonymity. Theywere informed that they could expect to commence treatment

within 6 weeks of the initial contact. To participate, patientssent in a signed informed consent form providing contact details.Study inclusion criteria were: (1) age 18-65 years, (2) access tothe Internet, and (3) a score between 14 and 29 on the BeckDepression Inventory-II (BDI-II), indicating mild to moderatesymptoms of depression. During the first months of the study,the protocol was changed by extending the inclusion criterionon the BDI-II to include participants with scores between 10and 40. This change was because of insufficient recruitmentand the clinical appraisal that patients with scores above 30could possibly benefit from the treatment based on their dailyfunctioning and motivation. In addition, their depression wastoo mild to assure them other public treatment options.Furthermore, several patients with a BDI-II score below 14reported a need for treatment. Based on this revised criteria, 7eligible patients were falsely excluded in the initial phase of thetrial. Individuals currently undergoing CBT were excluded,whereas individuals who used antidepressant medication werestabilized for 1 month prior to evaluation of diagnosticeligibility. To maximize the external validity of the trial, aheterogeneous group of patients with depressive symptoms wasincluded, independent of a particular diagnosis. Therefore,medical or psychiatric comorbidities only restricted inclusionwhen there was a need for immediate treatment of thesecomorbid conditions (suicidal ideation, current psychosis) or ifthe conditions were expected to markedly interfere withtreatment of the depressive condition (alcohol or drug usedisorders).

MeasuresAll outcome measures were based on self-report. Assessmentsof symptoms of depression, anxiety, and quality of life usingpaper-and-pencil questionnaires were completed by allparticipants at baseline, posttreatment, and at 6 monthsposttreatment (online questionnaires). The control group alsocompleted these inventories before entering online treatment(postwaiting). BDI-II was administered before everyconsultation during the intervention phase.

The primary outcome measure was the BDI-II, a 21-itemmeasure of severity of depressive symptoms during the past 2weeks [56]. Each item is rated on a 4-point scale ranging from0 to 3. Studies consistently support the BDI-II as a reliable,internally consistent, and valid scale for assessing depressionboth in psychiatric outpatients, the general population, and inprimary care settings [56-58]. Several studies have found highcorrelations between the paper-and-pencil and thecomputerized/online versions of the BDI-II [59-62]. In thepresent study, internal consistency (Cronbach alpha) was .78pretreatment, .93 posttreatment, and .94 at 6-month follow-up,respectively.

The secondary outcome measures consisted of the Beck AnxietyInventory (BAI), the Hospital Anxiety and Depression Scale(HADS), and 2 measures of quality of life—Satisfaction WithLife Scale (SWLS) and the EuroQol Group 5-DimensionSelf-Report Questionnaire (EQ-5D)—as well as a measure oftreatment satisfaction. The quality of life measures wereincluded during the initial phase of the trial considering that the



XSL•FORenderX



extension of outcomes beyond symptom measures wouldstrengthen the study.

The BAI is a 21-item measure of anxiety symptom severity[63]. Each item is rated from 0 to 3 depending on symptomseverity during the past week. The inventory possesses highinternal consistency and reliability, as well as robust convergentand discriminant validity [64-66]. Equivalent psychometricproperties have been shown across paper-and-pencil and onlineformats of the questionnaire, and the 2 formats are highlycorrelated [59,67]. Cronbach alphas in the present study were.93 at pretreatment, .88 at posttreatment, and .92 at 6-monthfollow-up, respectively.

The HADS is a 14-item inventory with 2 subscales of 7 itemseach, measuring depression and anxiety, respectively [68]. Eachitem is rated on a 0 to 3 scale. The inventory has good to verygood construct validity and internal consistency [68-70]. Mostfactor analyses confirm a 2-factor solution comprising adepression and an anxiety subscale [69]. Paper-and-pencil andInternet administration of the measure yields comparablepsychometric properties, but Internet administration mayoverestimate scores [70,71]. In the present study, Cronbachalpha was .68 and .82 at pretreatment, .82 and .84 atposttreatment, and .87 and .86 at 6-month follow-up for thedepression and anxiety subscales, respectively.

The EQ-5D is a generic questionnaire evaluating health-relatedquality of life [72]. The respondent marks his/her level offunctioning (no problems, some problems, extreme problems)for each of 5 dimensions (mobility, self-care, usual activities,pain/discomfort, and anxiety/depression) and rates his/her healthstate on a visual analog scale (EQ VAS) with the endpointslabeled best imaginable health state and worst imaginable healthstate, respectively. For the 5 dimensions, a scoring algorithm(the MVH_A1 tariff) based on preference weights was used toaggregate an index score (EQ Index) [73]. The health states“perfect health” (no problems on any dimension) and “dead”are assigned the values of 1 and 0, respectively. The instrumenthas been demonstrated to discriminate between subgroups ofpatients with differing severities of mental illness and to capturechanges in quality of life associated with improved mental healthover time [74]. No significant differences have been foundbetween scores obtained using paper and computerized modesof administration [75,76].

The SWLS measures global life satisfaction as acognitive-judgemental process, in which individuals assess theirquality of life according to their own criteria [77]. Therespondent rates on a 7-point Likert scale the degree to whichhe/she agrees with 5 statements. Several studies confirm thescale’s unidimensional structure and support its soundpsychometric properties, including internal consistency,test-retest reliability, as well as convergent and discriminantconstruct validity [78-80]. Research indicates that Internet dataon this measure is as reliable and valid as paper-and-pencil data[81]. Cronbach alpha in this study was .79 at pretreatment, .87at posttreatment, and .93 at 6-month follow-up.

The Mini-International Neuropsychiatric Interview (MINI) isa short, structured diagnostic interview for identifying thediagnoses of the Diagnostic and Statistical Manual of Mental

Disorders (Fourth Edition; DSM-IV) and the InternationalClassification of Diseases, Tenth Revision (ICD-10). It consistsof 17 modules. A comparison of MINI with other structuredclinical interviews shows sensitivities and specificities above0.70 for most diagnoses [82]. Excellent interrater reliability hasbeen reported. The MINI Interview was used to determinepsychiatric comorbidity and for excluding participants withcurrent psychosis or suicidal ideation, as indicated by 17 pointsor more on the suicidal ideation module.

The Alcohol Use Disorders Identification Test (AUDIT) is ascreening instrument consisting of 10 questions about alcoholuse in the past 12 months, alcohol dependence symptoms, andalcohol-related problems [83]. Eight items are rated on a 5-pointscale (0-4) and 2 items are rated on a 3-point scale (0, 2, 4). Alarge body of research confirms the favorable internalconsistency, reliability, and criterion validity [83-85]. In thisstudy, the scale was used to screen for alcohol use problems. Acutoff score of 20 was chosen to exclude patients in need offurther diagnostic evaluation for alcohol dependence [85]. Forparticipants scoring above16, alcohol use was monitored duringtreatment.

The Drug Use Disorders Identification Test (DUDIT) is an11-item screening instrument measuring patterns of substanceuse during the past 12 months, as well as various drug-relatedproblems [86]. Nine items are rated on a 5-point scale (0-4) and2 items are rated on a 3-point scale (0, 2, 4). In a sample of drugusers, the scale has shown good reliability, and it predicts drugdependence with a sensitivity of 0.90 and average specificityof approximately 0.80 [86]. In the present study, the DUDITwas used to screen for drug use disorder. A cutoff score of 25was used to exclude patients with a high probability of drugdependency.

Satisfaction with treatment was measured by 9 questions thatrespondents rated on a 5-point scale (1-5, very negative to verypositive). The questions concerned their satisfaction with theintervention as a whole and various aspects of the self-helpprogram and follow-up sessions. The general content of thequestions was influenced by patient satisfaction questionnairesused in other studies [87-89]. However, the exact content wastailored to tap into aspects of treatment considered importantfor the purpose of the present investigation. The questions aredescribed in detail in the Results section.

Treatment variables included module completion, number ofsessions, treatment duration, session duration (in minutes, notincluding screening), and total time spent by therapists (timespent outside the consultations was not registered). User dataon module completion was registered online and was denotedby a number between 0 and 4, with 0 indicating no use and 4indicating completion of the module. For the variable time spentby therapists, the amount of missing data was considerable(51.9% for time spent on screening, 14.6% for total time); thus,these data can only be considered estimates. Total time wasestimated by imputing mean screening time for missing dataconcerning screening duration and each individual’s meansession duration for missing data from treatment sessions.



XSL•FORenderX



ProcedureAfter informed consent was obtained, participants were screenedfor inclusion through a face-to-face session. A computerizedrandom number generator randomized identification (ID)numbers to the 2 groups (generated by KL). Eligible participantswere given ID numbers following a chronological sequence.To ensure equal group sizes, blocked randomization withvariable block sizes were used. Patients could not be blindedto group assignment, but were blinded to the status of the waitlistas a control condition.

Screening, enrollment, and treatment were carried out by 2licensed clinical psychologists (RSH and KL) with basic CBTskills and good knowledge of the MoodGYM program. Bothhad less than 2 years of experience in clinical practice and noprior experience with Internet-based treatment. The therapistswere not blind to the participants’ group allocation. However,steps were taken to blind the evaluation of outcomes by ensuringthat posttests were performed by a research assistant unawareof the participants’ allocation assignment.

InterventionParticipants in both groups were free to access usual primarycare treatment, which could include antidepressant medication,informal supportive therapy, or referral to specialist mentalhealth services.

The guided self-help intervention involved 3 components: (1)The Norwegian version of the Web-based CBT programMoodGYM version 3 [90], (2) brief face-to-face therapistsupport, and (3) tailored emails between sessions. MoodGYMwas originally developed at the Australian National Universityto prevent depression in young people aged between 15 and 25years. However, data from individuals who used the Englishversion of the program has shown that most users were aged25 to 44 years, and that the users’ average depression andanxiety scores were elevated compared to the general population[91]. Therefore, the program appears useful for older age groupsthan originally targeted, and for individuals with elevated levelsof depressive and anxious symptomatology. The programconsists of 5 modules and a personal workbook containingexercises and assessments. Module 1 through 3 focus on thecognitive model, typical patterns of dysfunctional thinking, andexercises to identify and restructure dysfunctional thinking, aswell as behavioral strategies to increase engagement in positiveactivities. Module 4 focuses on stress and stress reduction andintroduces relaxation techniques. Module 5 covers simpleproblem solving and typical responses to broken relationships.Each module takes approximately 45 to 60 minutes to workthrough. See Figure 1 for screenshots from the program.

In the first session after screening, participants were introducedto the program, received their trial username and password, andwere instructed to work at home with 1 module each week.After each module, participants received face-to-face support(15-30 minutes). The therapists followed a guideline script with3 compulsory topics for every consultation: (1) monitoring ofdepression symptoms and discussion of changes, (2) a focus onthe important topics and exercises covered by each module andthe participants’ experiences of working with it, and (3)

introduction of the next module and motivate participants toadhere to the treatment plan. The main focus of the therapistwas on reinforcing the efforts made by participants and helpingthem to relate to the material and to incorporate the use oftechniques from the program into their everyday living. If timepermitted, participants could also bring up other topics theyconsidered important in relation to their depression. In theconcluding session, the experiences and outcomes of treatmentwere discussed. Therapists aimed to meet participants weeklyand to complete the intervention over 7 weeks. However, theinterval between sessions and the number of sessions wereallowed to vary somewhat to provide flexibility in meetingindividual needs. Between sessions, participants receivedtailored emails aiming to motivate them to work with theself-help program. The emails introduced the next module, andsome contained brief advice on how to overcome depressivesymptoms (eg, the importance of behavioral activation).Participants did not get a mental health record at the clinic, buta short case summary was sent to their GP (with consent fromthe participants). Participants considered in need of moreextensive treatment throughout or after completing the trialwere assisted in the process of referral to specialized mentalhealth services. The Web-based program did not store anypersonally identifying information about users.

Statistical AnalysesAll analyses were carried out using IBM SPSS Statistics version19 for Windows (IBM Corp, Armonk, NY, USA), except forthe power calculation which was performed using G*Power(Heinrich-Heine-Universität, Institut für ExperimentellePsychologie, Düsseldorf, Germany).

Differences between the groups on baseline characteristics wereexamined by performing chi-square tests for categoricalvariables and 1-way analysis of variance (ANOVA) forcontinuous variables. A logistic regression analysis withbackward stepwise method was used to explore whether missingdata at postintervention (not completing postinterventionmeasures) could be significantly predicted by participantcharacteristics.

Results on the BDI-II and BAI were analyzed usingintention-to-treat (ITT) analyses in which participants areanalyzed in the group they were randomized to, irrespective oftreatment adherence. Because of missing data at pretest for theremaining secondary measures (3% missing on the HADS, and17% and 19% missing on SWLS and EQ-5D, respectively),modified ITT analysis was performed including all participantscompleting the measures at least once. Effects were tested byperforming linear mixed-models analysis using the restrictedmaximum likelihood (REML) estimation procedure and anunstructured covariance matrix. Since linear mixed-modelsanalysis can handle incomplete data, no procedure for imputationof missing data was utilized in the analysis [92]. For the analysisof BDI-II during the treatment phase, random intercepts acrossparticipants were estimated, and BDI-IIs from every treatmentsession were included for the intervention group. Time wascoded as 0 for baseline and as number of weeks from baselinefor all subsequent measures. To control for differences intreatment duration, the time frame was made comparable



XSL•FORenderX



between groups by including only measures up to 7 weeks afterbaseline (the planned time frame for completing theintervention) for the intervention group in the main analysis.For the secondary measures and for analysis including the6-month follow-up data on the BDI-II, repeated measures linearmixed-models analysis was performed with occasion (baseline,posttest, 6-month follow-up) as the repeated factor. Thisprocedure was deemed acceptable because linear regressionanalyses did not find treatment duration to be a significantpredictor of symptom change during the treatment phase of theintervention group for any of the secondary measures (beta=-.16to .28, t29-35=–0.91 to 1.58, P=.12-.92). Scores on the last BDI-IIfrom participants completing 5 or more weeks of treatment butmissing formal posttest data (n=8) were included in the analysisbecause this was considered to give a more accurate estimateof change over time.

For completers, analyses of covariance (ANCOVA) wereperformed with postsymptom scores as the dependent variableand preintervention symptom scores and treatment duration ascovariates. Effect sizes (Cohen’s d) were calculated for within-and between-group changes based on estimated means or beta

coefficients [93]. For the ITT analyses, calculations were basedon pooled standard deviations calculated from the square rootof each groups variance parameters from the mixed-modelsanalysis: the single variance estimate of the procedure with arandom intercept, and the sum of the variance estimates at eachtime point of interest minus 2 times the covariance estimatebetween these time points for the repeated procedure [94,95].For the completer analyses, the square root of the mean squareerror, equivalent to the pooled standard deviation, was used. ACohen’s d of 0.2 was considered a small effect, 0.5 a mediumeffect, and 0.8 or more a large effect [96].

Clinically significant changes on the BDI-II were assessed usingthe criteria for reliable change and cutoff points developed forthe BDI by Seggar et al [97], based on the definition by Jacobsonand Truax [98]. Recovery was defined as the combination ofreliable improvement (a change of more than 8.5 points on theBDI-II) and endpoint symptom level below the clinical cutoffof 14.3. For those with subclinical symptoms at baseline, reliableimprovement required a change of more than 4.6 points andrecovery required reliable improvement plus an endpoint lessthan 4.1 [97]. Change scores between baseline and 7 weeks oftreatment were used for the intervention group.

Figure 1. Screenshots from MoodGYM.



XSL•FORenderX



Results

Sample CharacteristicsFigure 2 shows the flow of participants through the trial. Of the128 individuals screened for participation, 106 (83%) werefound eligible. Most participants, including 49 of 52 (94%) inthe intervention group and 48 of 54 (89%) in thedelayed-treatment group, were recruited from GPs. Theremaining participants were recruited from waitlists at 2outpatient clinics (n=3), from primary care nurses (n=4), andfrom a low-threshold clinic for youth (n=2). Postinterventionmeasures were completed by 71% (37/52) and 87% (47/54) ofthe participants in the intervention and the delayed-treatmentgroup, respectively. The 6-month follow-up assessment wascompleted by 42 participants (81%) in the intervention groupand by 34 participants (63%) in the control group.

Group and educational level emerged as significant predictorsof dropout at postintervention. The odds of dropping out beforethe posttest was significantly higher for participants in theintervention group relative to participants in the

delayed-treatment group (OR 3.03, 95% CI 1.08-8.47, P=.04),and significantly lower for participants with higher educationrelative to participants with a lower educational level (OR 0.36,95% CI 0.13-0.99, P=.048). No other demographic or clinicalvariables predicted dropout at postintervention.

Table 1 shows the descriptive characteristics of the sample. Formost demographic and clinical variables the 2 groups did notdiffer significantly at baseline (P=.10-.90). However, the groupsdiffered significantly with regard to age (P=.045), with theintervention group being slightly older than the control group.The groups also differed on the variable comorbid anxiety(P=.03), with the number of participants with an anxiety disorderbeing significantly higher in the control group compared to theintervention group. Baseline scores on all symptom and outcomemeasures were comparable between the groups (P=.13-.87).

Further exploration of the 2 groups with regard to anxiety levelas measured with BAI shows that although a higher proportionof the control group had symptoms corresponding to moderateor severe anxiety compared to the intervention group, thedistribution across the categories minimal, mild, moderate, andsevere anxiety did not differ significantly (P=.29).

Figure 2. Flow of participants through the trial.



XSL•FORenderX



Table 1. Participant characteristics at baseline.

Total (N=106)Waitlist (n=54)Intervention(n=52)

Participant characteristics

Gender, n (%)

29 (27.4)14 (25.9)15 (28.8)Male

77 (72.6)40 (74.1)37 (71.2)Female

Age (years)

36.1 (11.3)a33.9 (9.9)38.3 (12.2)Mean (SD)

18 - 6318 - 5819 - 63Range

Marital status, n (%)

59 (55.7)31 (57.4)28 (53.8)Married/living together

6 (5.7)4 (7.4)2 (3.8)Separate living

8 (7.5)5 (9.3)3 (5.8)Divorced

33 (31.1)14 (25.9)19 (36.5)Single

Highest educational level, b n (%)

7 (6.6)1 (1.9)6 (11.5)Compulsory school

(9 or 10 years)

42 (39.6)26 (48.1)16 (30.8)High school

31 (29.2)14 (25.9)17 (32.7)University, 3 years

25 (23.6)12 (22.2)13 (25.0)University, ≥5 years

Employment, n (%)

74 (69.8)37 (68.5)37 (71.2)Employed (full- or part-time)

11 (10.4)6 (11.1)5 (9.6)Student

10 (9.4)3 (5.6)7 (13.5)Long term sick

6 (5.7)5 (9.3)1 (1.9)Homemaker

5 (4.7)3 (5.6)2 (3.8)Unemployed

39 (52.7)c21 (56.8)18 (48.6)Sick leave (employed sample)

Present treatment, d n (%)

20 (18.9)7 (13.0)13 (25.0)Medication

11 (10.4)4 (7.4)7 (13.5)Other treatment

80 (75.5)44 (81.5)36 (69.2)None

Treatment history, e n (%)

62 (58.5)30 (55.6)32 (61.5)Earlier

41 (38.7)23 (42.6)18 (34.6)None

51 (48.1)25 (46.3)26 (50.0)Depression current,fn (%)

Number of major depressive episodes, g n (%)

11 (10.4)6 (11.1)5 (9.6)0

34 (32.1)18 (33.3)16 (30.8)1

29 (27.4)15 (27.8)14 (26.9)2 - 4

25 (23.6)11 (20.4)14 (26.9)≥5

Comorbidity, h n (%)

35 (33.0)a23 (42.6)12 (23.1)Anxiety



XSL•FORenderX



Total (N=106)Waitlist (n=54)Intervention(n=52)

Participant characteristics

5 (4.7)4 (7.4 %)1 (1.9)Other axis-I disorder

Symptom measures, i mean (SD)

5.1 (4.0)5.4 (4.3)4.8 (3.8)AUDIT

0.6 (2.3)0.7 (2.5)0.4 (2.2)DUDIT

Internet Use, j n (%)

92 (86.8)42 (77.8)50 (96.2)Daily

3 (2.8)3 (5.6)0 (0.0)Weekly

aP<.05.b0.9% missing.c36.8% of the total sample.dMedication is antidepressants; other treatment is psychological therapy other than CBT.e2.8% missing.fMajor depressive episode fulfilling DSM-IV criteria.g6.6% missing.hAnxiety includes panic disorder, agoraphobia, social phobia, and generalized anxiety; other axis-1 disorders include bipolar disorder, obsessive-compulsivedisorder, bulimia, and posttraumatic stress disorder.iAUDIT: Alcohol Use Disorders Identification Test, 0.9% missing; DUDIT: Drug Use Disorders Identification Test, 0.9% missing.j10.4% missing.

Attrition and AdherenceOf the 52 participants in the intervention group, 31 (60%)adhered to the treatment program in that they completedMoodGYM and attended at least 7 sessions. Total nonadherencewas 40% (21/52). Reasons for nonadherence are summarizedin Figure 2. Overall, the sample starting treatment (n=50)completed a mean of 3.8 (SD 1.7) of the 5 modules, attended amean of 7.2 (SD 2.3) sessions, with average session durationof 27.7 (SD 6.2) minutes. The average number of weeks intreatment was 11.3 (SD 7.2). Total time spent by therapistsranged from 70 to 506 minutes (mean 242.1, SD 96.6). Aprevious study found that versions of the MoodGYM programencompassing Module 2 (extended CBT) were associated withgreater improvements than versions excluding this module [91].This suggests that Module 2 may be a particularly importanttreatment component. Of the 50 participants starting treatment,86% (n=43) completed 2 or more modules, indicating that theymay have completed enough of the treatment program togenerate beneficial outcomes.

Depression and Anxiety SymptomsTable 2 depicts the preintervention, postintervention, and6-month follow-up means and standard deviations for eachgroup, as well as within-group and between-group effect sizes.The ITT analysis for the primary outcome measure, BDI-II,revealed a significant time by treatment group interaction(F1,244.83=9.55, P=.002, d=0.65). There was also a significanteffect of time (F1,245.37=11.87, P=.001). Both groups experiencedsignificant improvements of depressive symptoms during thetreatment phase, but this improvement was significantly largerin the intervention group compared to the delayed-treatmentgroup. Because most of the intervention group had not yetcompleted treatment at 7 weeks, the analysis was repeated using

scores from the intervention group up to 8, 10, 12, and 14 weeks.The interaction term remained significant(F1,268.81-333.62=6.34-10.88, P=.01-.001, d=0.54-0.67). Repeatedlinear mixed-models analysis also found significant differenttime trends for the groups between posttest and 6-monthfollow-up (t81.17=2.88, P=.005). During this time, thedelayed-treatment group had received treatment, and pairwisecomparisons indicated a significant decrease in symptoms inthis group (P=.001), whereas level of depressive symptoms inthe intervention group remained stable (P=.56).

The ITT analysis for the BAI revealed no significant interactionbetween treatment group and occasion (F1, 84.31=0.37, P=.69).Pairwise comparisons showed that both groups improvedsignificantly between baseline and posttest (P=.007 and P=.02for the intervention and control group, respectively; see Table2). Between posttest and 6-month follow-up, the control groupfurther improved (P=.04), whereas the intervention groupremained unchanged (P=.36). Analysis for the HADS subscalesyielded significant group by occasion interactions for both thedepression subscale (F1,78.05=14.68, P<.001) and the anxietysubscale (F1,78.07=8.10, P=.001). Pairwise comparisons for bothsubscales found that the intervention group, but not the controlgroup, reduced their scores significantly between pretest andposttest (P<.001 and P=.46, respectively, for the depressionsubscale and P<.001 and P=.44, respectively, for the anxietysubscale; Table 2). Between posttest and 6-month follow-up,the control group experienced a significant reduction in bothdepressive and anxiety symptoms (P=.001 and P<.001,respectively), whereas scores did not change significantly forthe intervention group for the depressive subscale (P=.07) orfor the anxiety subscale (P=.80).



XSL•FORenderX



For all reported ITT and modified ITT analyses, baseline anxietysymptoms on the BAI and age were included as covariates, withthe exception that baseline BAI scores were not included as acovariate in the analysis of BAI. The effect of BAI scores wasconsistently significant (P=.02 to P<.001). The effect of agewas significant in 2 of 8 models (P=.008-.03).

Completer AnalysesThe results of the ANCOVA revealed the same pattern of results.There was a significant effect of group on posttest level ofdepressive symptoms measured with BDI-II (n=84; P<.001,d=1.09) and with HADS (n=76; P=.002, d=0.95), and level ofanxious worry as measured with HADS (n=76; P=.002, d=0.97)after controlling for the effect of baseline symptoms andtreatment duration. The results were not significant for anxietyas measured with BAI (n=83; P=.47, d=0.20). Thus, fordepression and anxiety measured with HADS, the groupsdiffered significantly in posttest scores after controlling fordifferences in preintervention scores and treatment duration.Baseline symptoms were significantly related to posttestsymptoms for all measures (P<.001), whereas treatment durationdid not significantly affect outcome (P=.19-.92).

Quality of LifeFor the SWLS, the modified ITT analysis showed a significantinteraction between treatment group and occasion (F1,75.75=8.49,P<.001). Pairwise comparisons found a significant increase insatisfaction for the intervention group from pretest to posttest(P<.001), but no such change for the delayed-treatment group(P=.52; see Table 2). Between posttest and 6-month follow-up,there was a significant increase in life satisfaction in the controlgroup (P=.01), whereas the intervention group did notexperience significant changes (P=.08). The analyses for EQIndex yielded an overall significant difference in time trendsbetween the groups (F1,67.42=3.55, P=.03). Between pretest andposttest, there was no significant interaction between treatmentgroup and occasion (t68.82=–1.00, P=.32), with both groupsimproving at comparable rates over time. However, there wasa significant group by occasion interaction between posttest and6-month follow-up (t63.96=–2.66, P=.01, with the control groupshowing significant improvement (P=.02), whereas there wereno significant changes in the intervention group (P=.18). Forthe EQ VAS, the overall interaction between group and occasionwas not significant (F1,71.32=2.25, P=.11). Between pretest and

posttest both the intervention group (P<.001) and the controlgroup (P=.03) experienced significantly improved self-reportedhealth state (Table 2), but the interaction between group andoccasion was not significant (t66.07=–1.94, P=.06). Betweenposttest and 6-month follow-up, pairwise comparisons showeda slight, but nonsignificant, improvement in the control group(P=.06), whereas scores in the intervention group remainedstable (P=.74).

Completer AnalysesSimilar results were found for the completer analyses, in whichANCOVA showed a significant effect of group for the SWLS(n=66) after controlling for baseline levels of life satisfactionand treatment duration (P=.006, d=0.86). There was nosignificant effect of group on health-related quality of life atposttest, EQ Index (n=63; P=.56, d=0.18), or health state atposttest, EQ VAS (n=61; P=.11, d=0.52), despite a moderateeffect size for the latter. The effect of baseline scores wassignificant for all measures (P≤.002). There were no significanteffects of treatment duration (P=.14-.99).

Clinical Significance of Changes in DepressiveSymptomsTable 3 presents data on clinically significant change on theBDI-II, based on scores at 7 weeks for the intervention group.The ITT procedure was used (classifying all who did not starttreatment or did not complete their waitlist period asnonresponders). The results of the chi-square tests for the fullsample and the sample with BDI-scores above clinical cutoffshow that significantly more participants recovered in theintervention group compared to the delayed-treatment controlgroup. Conversely, a significantly smaller proportion of theintervention group experienced no change within theintervention period. For the sample fulfilling the criteria for amajor depressive episode at baseline, the same trend was evidentbut the difference in rate of recovery did not reach significance.The rates of improvement and deterioration were similar in the2 groups for all analyses. The same analysis carried out afterexcluding participants (n=8) who during the waitlist orintervention period started or increased their dosage ofantidepressant medication, or commenced other psychologicaltreatment, produced similar patterns of results. At 7 weeks oftreatment 37 of 52 participants (71%) in the intervention grouphad completed 2 or more modules, whereas 15% (n=8) hadcompleted treatment.



XSL•FORenderX



Table 2. Estimated means (EM),a observed means (OM), observed standard deviations (SD), standard deviations based on linear mixed-models varianceestimates(SDm), and effect sizes from pretreatment (pre) to posttreatment (post) and pretreatment to 6-month follow-up (6 m) for the intervention andthe delayed-treatment control group.

Effect size (Cohen’s d)bDelayed treatment (n=54)Intervention (n=52)Measures

Pre-6 mPre-postSDmSDOMEMSDmSDOMEM

WdtWiBWdtWiB

–1.02–0.81–0.12–0.65–0.980.65BDI-II

6.7422.2721.856.8521.1321.37Pretreatment

4.638.6418.6319.075.868.1514.20c15.15c7 weeks

9.5610.9812.8211.8610.359.3212.4513.396 months

–0.65–0.48–0.12–0.35–0.410.08BAI

10.9015.3315.1011.1012.0512.23Pretreatment

8.038.1012.8312.308.219.268.368.80Posttreatment

9.059.799.419.1910.376.617.077.466 months

–0.65–0.53–0.10–0.11–1.171.10HADS Depression

3.137.617.402.928.088.21Pretreatment

2.703.637.197.073.132.614.244.67Posttreatment

4.163.774.854.664.294.055.765.916 months

–0.55–0.52–0.130.13–0.600.74HADS Anxiety

4.599.599.153.958.819.14Pretreatment

3.044.1810.079.523.293.696.747.15Posttreatment

4.884.796.946.403.994.166.576.996 months

0.520.35–0.120.120.850.85SWLS

5.7516.3616.835.2516.5416.41Pretreatment

3.695.0217.2117.284.606.0421.4620.38Posttreatment

5.526.9120.0019.796.426.6319.0018.666 months

0.550.71–0.010.420.790.47EQ-5D VAS

17.8754.9356.0118.6859.1358.59Pretreatment

13.1017.0960.7761.6716.3515.3073.8871.13Posttreatment

21.5620.8666.4167.7816.0414.6571.7970.126 months

0.740.26–0.330.260.590.22EQ-5D Index

0.250.600.610.230.630.63Pretreatment

0.270.240.670.680.240.170.800.75Posttreatment

0.210.200.770.780.300.260.720.706 months

aEstimated means (except for BAI) are adjusted for the covariates baseline BAI score and age.bB: between-group effect size; Wi: within-group effect size for the intervention group; Wdt=within-group effect size for the delayed-treatment group.cEstimated mean after completing treatment=12.43, observed mean after completing treatment=11.34.



XSL•FORenderX



Table 3. Proportion of participants reaching the criteria for clinically significant improvement on the Beck Depression Inventory-II (BDI-II) at 7 weeks

of treatment and results of chi-square tests (χ2).

Current major depressive episode diagnosis,n (%) (n=51)

Baseline BDI-II above clinical cutoff,an(%) (n=90)Full sample, n (%) (N=106)

Treatment re-sponse

χ21

Control(n=25)

Intervention(n=26)χ2

1

Control(n=48)

Intervention(n=42)χ2

1Control (n=54)Intervention(n=52)

2.73 (12.0)8 (30.8)8.3d5 (10.4)15 (35.7)8.8b5 (9.3)17 (32.7)Recovered

1.13 (12.0)6 (23.1)0.34 (8.3)5 (11.9)0.95 (9.3)8 (15.4)Improved

4.6c18 (72.0)11 (42.3)8.4b38 (79.2)21 (50.0)7.7b41 (75.9)26 (50.0)No change

0.0011 (4.0)1 (3.8)0.011 (2.1)1 (2.4)0.33 (5.6)1 (1.9)Deteriorated

aBDI-II>14.bP<.01.cP<.05.

Therapist EffectsTherapist effects were investigated by looking at the interactionbetween therapist and time for the intervention group. Theanalyses indicated a significant difference between the 2therapists when analyzing BDI-II scores up to 7 and 8 weeksof treatment (P=.03-.04). This effect no longer reachedsignificance when including scores up to 10, 12, and 14 weeksof treatment (P=.05-.32). The analyses did not yield differentialtreatment effects across the 2 therapist for the HADS depressionsubscale (P=.87), nor for any other outcome measure(P=.50-.94). An exploratory linear regression analysis showedthat symptom change in the intervention group was notsignificantly predicted by the total time spent by the therapistsfor any outcome measure (beta=–.12 to .26, t29-48=–0.81 to 1.52,P=.14-.96).

Treatment SatisfactionTable 4 shows the response frequencies for questions regardingsatisfaction with the treatment. The results are reported forparticipants in both groups (intervention group: n=39;delayed-treatment control group: n=26) who completed the fulltreatment or parts of it. Overall satisfaction with the treatmentwas high, with 89% (58/65) giving the intervention as a wholea rating of 4 or 5 on a scale with 5 being very satisfied (seeTable 4). Most participants also indicated that they wouldrecommend the combined intervention to a friend with a similarproblem. The ratings of the MoodGYM program were positive,

but somewhat more moderate with between 50% and 60% givingclearly positive ratings (4 or 5 on the 5-point scales, see Table4) to the benefit of the program, the usefulness of the exercises,and the relevance of the thematic content, and none rating theprogram as not useful or relevant. The benefit of the treatmentsessions and the relationship with the therapist were ratedpositively by more than 90% (60/65 and 64/65, respectively)of the participants.

Service Use and Work Status After TreatmentOf the 76 participants who completed the follow-up assessment,45% (19/42) of participants in the intervention group and 38%(13/34) of participants in the control group had receivedtreatment for mental health problems during the 6-monthfollow-up period. Two participants (3%) had been hospitalized,19 (25%) had used antidepressant medication (15 currentlyusing), 26 (34%) had received psychological treatmentindividually or group therapy, and 16 (21%) had receivedtreatment from their GP. Of the 19 participants reporting useof antidepressants, only 6 had commenced this treatment duringthe follow-up period.

With regard to work status, 6 of 42 respondents (14%) in theintervention group reported that they had been on sick leave atsome point in the follow-up period due to feeling tired, stressed,or experiencing mental health problems, whereas 9 of 34respondents (26%) in the control group reported sick leaveduring this period.



XSL•FORenderX



Table 4. Response frequencies regarding satisfaction and experiences with the treatment (n=65).

Satisfaction/experience scaleItem

54321

Overall satisfaction with the treatment

Very satisfiedVery dissatisfiedScale

18 (27.7)40 (61.5)7 (10.8)0 (0.0)0 (0.0)n (%)

Change in symptoms

Much improvedNeither norMuch worseScale

21 (32.3)26 (40.0)16 (24.6)2 (3.1)0 (0.0)n (%)

Would recommend the treatment to a friend with a similar problem

Yes, definitelyDefinitely notScale

40 (61.5)21 (32.3)4 (6.2)0 (0.0)0 (0.0)n (%)

Benefit of using MoodGYM

Highly beneficialNo benefitScale

5 (7.7)32 (49.2)24 (36.9)4 (6.2)0 (0.0)n (%)

The usefulness of the exercises in MoodGYM

Very usefulNot usefulScale

9 (13.8)28 (43.1)23 (35.4)5 (7.7)0 (0.0)n (%)

Relevance of the thematic content of MoodGYM a

Highly relevantNo relevanceScale

I: 5 (12.8) C: 6(23.1)

I: 10 (25.6) C: 13(50.0)

I: 20 (51.3) C: 6(23.1)

I: 4 (10.3) C: 1(3.8)

0 (0.0)n (%)

Benefit of the follow-up sessions

Highly beneficialNo benefitScale

26 (40.0)34 (52.3)5 (7.7)0 (0.0)0 (0.0)n (%)

Satisfaction with the number of sessions

Too manyJust enoughToo fewScale

0 (0.0)3 (4.6)55 (84.6)6 (9.2)1 (1.5)n (%)

Relationship to the therapist

Very positiveVery negativeScale

50 (76.9)14 (21.5)1 (1.5)0 (0.0)0 (0.0)n (%)

aP<.05, frequencies reported separately for the intervention (I) and delayed-treatment control (C) groups.

Discussion

Principal FindingsThe results of the present study indicate that a guided self-helpintervention combining the MoodGYM program withface-to-face therapist support can be effective in reducingdepressive symptoms for a sample of mildly to moderatelydepressed individuals recruited from primary care. Theintervention also had significant positive effects on symptomsof anxious worry, and participants experienced significantimprovements in global satisfaction with life. At 6-monthfollow-up, positive gains in terms of reduction of depressiveand anxious symptoms were largely maintained, whereasimprovements in life satisfaction were partly maintained. The

rate of nonadherence (40%) was moderate and the evaluationsof the treatment as a whole were predominately positive.

These findings are consistent with previous research in whichfavorable outcomes have been shown for treatments combiningMoodGYM and face-to-face support from a professional [29,37].The trials are not fully comparable, though, because the presentstudy used a delayed-treatment control condition, whereas bothprevious trials used comparison groups that received more activetreatments. This makes direct comparisons of between-groupeffect sizes difficult. However, the effect of guided self-helpfor mild to moderate depression using other Internet-basedprograms has generally been in the moderate to large range[19,27]. The magnitude of the between-group effect sizes ondepression measured with BDI-II in this study was within thisrange, but somewhat below average, whereas the effects on



XSL•FORenderX



SWLS and on depression and anxiety measured with HADSwere commensurable to effects found previously for similarmeasures [12-14,27,99,100]. Also, the rates of recovery and ofimprovement and recovery combined (33% and 48%,respectively) are in good accordance with results of priorinvestigations, in which clinically significant improvement andrecovery have varied between 25% and 50% [12,14,15,30,100].The nature of the guidance provided in these studies is somewhatheterogeneous, with some studies, such as the present study,defining guidance as active engagement in the therapeuticprocess, whereas several other studies have focused primarilyon providing feedback and encouragement. However, there isno indication of significantly differential treatment effectsdepending on the nature of the guidance to date [19,100]. Thesomewhat smaller between-group differences of the presentstudy must be seen in relation to the relatively high degree ofpositive change in the control group, with effect sizes being inthe small to moderate range for several measures (see Table 2).Almost half of the sample did not fulfill the criteria for a majordepressive episode on enrollment in the trial, and previousstudies of minor depression have shown high rates of placeboresponse in primary care patients [101] and substantiallikelihood of spontaneous remission in the general population[102]. In the present study, the control group was also free toaccess usual treatment in general practice during the waitingperiod. In addition, prior to entering the study controlparticipants participated in a screening session in which theyhad the opportunity to describe their problems, something thatcould have a therapeutic effect per se. These factors may partlyexplain the positive gains in the delayed-treatment group and,hence, the modest differences in outcome between the groups.

The results of the 6-month follow-up are also encouraging inthat improvements of depression and anxiety symptoms werelargely maintained, and the number of participants reportingsick leave due to mental health problems was substantiallysmaller during follow-up compared to baseline (20% and 53%,respectively). The proportion of participants using antidepressantmedication at baseline and during the follow-up period wascomparable, but there was a considerable increase in the numberof participants who had accessed psychological treatment duringfollow-up (34%) compared to baseline (10%). This is, however,not surprising because many participants had already beenreferred to such treatment when entering the project, but maystill have contributed to further improvements and maintenanceof symptom reduction. For some measures, particularly thedepression subscale of the HADS and the SWLS, there was atendency toward an increase in symptoms and lowering of lifesatisfaction during the follow-up period. The inclusion of boostersessions after the completion of the active treatment phase couldbe a measure to accomplish continued use of helpful techniquesand skills and the prevention of symptom relapse. Furtherresearch is needed to clarify this issue.

The results of the present trial are also consistent with researchsuggesting that MoodGYM, despite its main focus on depressivethought content, can have significant positive effects on anxietysymptoms [29,35,103]. In the present study, significant treatmenteffects were found for anxiety symptoms measured with HADS,but not with BAI. The questions on the HADS focus primarily

on symptoms such as worry, nervousness, and not being ableto relax. This is in good accordance with the core symptoms ofthe Goldberg Anxiety Scale [104], which has been used inseveral studies with MoodGYM. In comparison, the BAIincorporates both these subjective anxiety symptoms, as wellas 3 more somatic symptom clusters: neurophysiological,autonomic, and panic [63]. The MoodGYM program focusesprimarily on restructuring dysfunctional thinking and does notinclude an introduction to the physiology of anxiety or othertreatment techniques for anxiety. Thus, the present results, withthe program showing effects on anxious worry but not on morephysiological symptoms, seem to be in-line with the thematicfocus of the program.

The adherence rate in the present study is comparable to thatobserved in other online guided interventions, in whichadherence varied between 55% and 75% [13,14,23,30,99]. Therates of adherence are also comparable to those seen in otherpsychotherapy research and in regular clinical practice[105-107]. This level of adherence in the current study, and thehigh proportion (89%) of completers reporting being satisfiedwith the treatment, points to the acceptability of the intervention.The evaluation of the MoodGYM program was somewhat moremoderate with between 50% and 60% allocating anunambiguous positive rating to the benefit and relevance of theprogram. This moderate level of satisfaction in the present adultsample may arise from the fact that the MoodGYM programwas originally targeted at youth and young adults. Although thetherapists emphasized the applicability of the principles for allage groups when introducing the intervention, and manyparticipants managed well to make use of the content,participants frequently characterized the program as “tooyoung.” As these aspects of treatment were not formallymeasured, these factors require further investigation to beproperly elucidated.

Strengths and LimitationsThe current study was designed to trial a treatment procedureprior to its evaluation in general practice. Therefore, we soughtto ensure a high level of internal validity while at the same timeaiming to increase external validity by reflecting theheterogeneity of patients in real clinical practice. One strengthof the study is the relatively heterogeneous sample ofparticipants with regard to the range of depression and anxietysymptoms. There was also substantial comorbidity with anxietydisorders, although lower than rates found in population-basedstudies [108,109]. The fact that 83% of screened participantswere found eligible also indicates that the sample isrepresentative of those who opted for this choice of treatment.

An overarching focus in designing the intervention wasfeasibility for implementation in general practice. Studies havesuggested that GPs may find the implementation of CBTtechniques too time-consuming [49,110]. Therefore, sessionswere primarily supportive and structured by the Web-basedprogram. To allow for flexibility and increase feasibility, aguideline script rather than a more comprehensive manualguided each consultation. This lack of rigid standardization mayhave introduced some variability in treatment fidelity.



XSL•FORenderX



Blinding of participants was not possible for obvious reasonsin this trial. However, the control group was blinded to the statusof the waitlist as a control condition. Waitlists for treatment isthe norm in Norwegian mental health care, and the short waitfor the present treatment compared to other treatment options,may have minimized negative effects (“nocebo effects”) in thecontrol group.

The present study also has several limitations that need to beaddressed. First, the design of the study with only 1 interventiongroup receiving a compound of intervention elements does notallow for tests of the specific contribution of MoodGYM andthe face-to-face consultations. Second, the lack of allocationconcealment and blinding, and the role of the first and secondauthors as therapists in the trial introduced a risk of bias thatmay have inflated the treatment effects. Unfortunately, resourceconstraints prevented the use of independent therapists. The useof self-report measures rather than therapist assessments doesalleviate this problem to some degree. Biased outcomeassessments were further prevented by ensuring that a researchassistant without knowledge of the participants’ conditionassignment collected posttest data. Third, the sole reliance onself-report is a limitation in itself. Independent preassessmentsand assessments by a clinician blinded to condition allocationwould have been preferable and would have strengthened theresults. Fourth, at preintervention there was a lack ofcomparability in diagnosed anxiety between the groups with asignificantly larger proportion of the control group fulfillingthe criteria for an anxiety disorder. Despite this difference indiagnosed anxiety, scores on 2 different anxiety scales (HADSand BAI) were not significantly different, which suggestscomparable anxiety levels in the groups. To minimize the effectof differences in anxiety, all primary analyses were controlledfor anxiety level as measured with BAI, for which the observeddifference was most significant. Fifth, the use of an unequalnumber of assessments of depressive symptoms in the 2 groups,with the intervention group having weekly assessments and thecontrol group only completing a pretest and posttest, may haveresulted in more favorable effects in the intervention groupbecause of measurement effects. Previous studies of nonclinicalsamples have indicated that scores on the BDI tend to decreasewith repeated administration [111,112]. Whether this holds forclinical samples is less certain. In this study, the effect ofrepeated measurement cannot be clearly distinguished from thetreatment effect. However, comparable effects were also foundfor symptoms assessed only preintervention and postinterventionin both groups. This indicates that the treatment had beneficialeffects over and above possible measurement effects. Still, inlight of this limitation, the results must be interpreted withcaution. Sixth, the use of different administration formats forthe assessments of the treatment phase and follow-up,(paper-and-pencil vs online questionnaires, respectively) canpotentially introduce measurement bias. Although the 2 formatscorrelate highly, a previous study reported a significantdifference in mean scores on the BDI-II and BAI, which makesswitching of formats problematic [59]. Despite this limitation,the results should not be considered weakened for most measuresbecause the direction of differences has generally suggestedthat online versions tend to inflate estimations of symptomseverity and lower ratings of quality of life [59,71,76], with the

exception of BAI, for which Carlbring et al [59] found thatmeans on the online version were lower compared to thepaper-and-pencil version. The reliability of the 6-monthfollow-up results for the BAI may, therefore, be limited.Seventh, the multiplicity of outcomes increases the risk of typeI errors. However, the main findings of the present trial wouldstill be significant when employing the Bonferroni correction.This indicates the robustness of the findings. Finally, althoughthe heterogeneity of the sample and the recruitment fromprimary care is a strength, the generalizability of the results isuncertain because the sample was a self-selected group. Basedon the notifications by the GPs when informing a patient of thestudy, the estimated uptake (meeting up for screening) was 39%(128 of 325 who received information), which is slightly greaterthan the median uptake for computerized CBT [55]. Consideringthe extra barriers imposed by the research activities, this uptakerate is relatively high and indicates the possible acceptabilityof this treatment among depressed primary care patients. It alsostrengthens the generalizability of the results, by indicating thatthe self-selected group may be representative of a considerableproportion of the targeted group of primary care patients.

Potential Clinical Implications and Further ResearchThe positive treatment effects found for the intervention in thepresent study are encouraging and suggest that this interventionmay have a potential for use in a stepped-care approach. Thedemand for mental health treatment is higher than what can bemet by the current number of trained clinicians [113]. Toincrease availability of treatment, beneficial interventions mustbe delivered as efficiently as possible to as many people aspossible. The present intervention is time-limited, and becausethe CBT elements are largely delivered by the program, primarycare therapists with some training in CBT and MoodGYM,should be able to provide adequate guidance. In fact, studiesshow that guidance may be delivered effectively not only bytrained clinicians, but also by mental health workers with limitedexperience and by computer technicians [114,115]. Thus,dissemination of the current intervention to regular primary carecould be a step toward increasing access to psychologicaltherapies. However, the moderate ratings of the benefit andrelevance of the content of the Web-based program by an adultpopulation points to the need for a variety of Web-based toolsto make such treatments acceptable for a wider audience.

For practical reasons, we chose to use psychologists for thisfirst evaluation. Therefore, further research is needed todetermine if the present intervention would be as effective andacceptable in regular clinical practice when delivered by GPsor other primary care therapists. It may also be noted that thepresent intervention was more time-intensive than most otherguided self-help interventions. However, since the role of theclinician was mainly supportive and facilitative and the maintherapeutic input was delivered through a standardized treatmentpackage, the intervention was regarded conceptually as a guidedself-help intervention [31]. Similar effects have been found forlow- and high-intensity guided Internet-based psychotherapy[19]. Further research should investigate if the presentintervention with more limited therapist support could yieldsimilar effects.



XSL•FORenderX



ConclusionDespite its limitations, the present study indicates that anintervention combining the MoodGYM program with therapistsupport can be an effective treatment of depression in a sampleof primary care patients. The intervention not only alleviatesdepressive symptoms, but also has positive and significanteffects on symptoms of anxious worry and global satisfactionwith life. Positive gains in terms of reduction of depressive andanxious symptoms were largely maintained at 6-monthfollow-up, and improvements in life satisfaction were partlymaintained. Moderate rates of nonadherence and predominately

positive evaluations of the treatment as a whole also indicatesthe acceptability of the intervention. The intervention wasdesigned to be suitable for implementation in primary healthcare, and could have a potential for use in a stepped-careapproach. However, further research is necessary to determinewhether it is equally effective when delivered in regular primaryhealth care and whether the inclusion of booster sessions couldfurther improve symptom maintenance. Further research is alsoneeded to investigate whether the intervention is truly acceptablefor the wider group of primary care patients and whether it isconsidered feasible and acceptable by GPs or other primary caretherapists.

AcknowledgmentsThis study was funded by the Research Council of Norway (196423/V50). MoodGYM was provided by the Australian NationalUniversity. The authors would like to acknowledge the contribution of Anthony Bennett, IT manager of the e-hub, the e-mentalhealth research and development group at the Centre for Mental Health Research at ANU; Kylie Bennett, Development Managerat e-hub, who provided technical expertise in administration of the trial; and Ada Tam, e-hub Project Officer at ANU, whoprovided technical support for users in the trial. We would also like to thank Jan Abel Olsen at the Institute of CommunityMedicine at the University of Tromsø for contributing with his expertise on the EQ-5D scale. Last, but not least, we would liketo give a special thanks to those participating in the trial.

Conflicts of InterestKG is one of the authors of the MoodGYM intervention evaluated in the trial, and works for The Australian National Universitywho provides free access to the MoodGYM program. KW and ME contributed in the process of translating MoodGYM intoNorwegian. RSH, KL, TW, and NK have no financial or nonfinancial interests to declare in relation to this study.

Multimedia Appendix 1CONSORT-EHEALTH checklist V1.6.2 [53].

[PDF File (Adobe PDF File), 999KB - jmir_v15i7e153_app1.pdf ]

Multimedia Appendix 2Trial Protocol as requested in CONSORT-EHEALTH [53].


Multimedia Appendix 3Trial information to participants as requested in CONSORT-EHEALTH [53].


References1. Ayuso-Mateos JL, Vázquez-Barquero JL, Dowrick C, Lehtinen V, Dalgard OS, Casey P, ODIN Group. Depressive disorders

in Europe: prevalence figures from the ODIN study. Br J Psychiatry 2001 Oct;179:308-316 [FREE Full text] [Medline:11581110]

2. Wittchen HU, Jacobi F. Size and burden of mental disorders in Europe--a critical review and appraisal of 27 studies. EurNeuropsychopharmacol 2005 Aug;15(4):357-376. [doi: 10.1016/j.euroneuro.2005.04.012] [Medline: 15961293]

3. Kringlen E, Torgersen S, Cramer V. A Norwegian psychiatric epidemiological study. Am J Psychiatry 2001Jul;158(7):1091-1098. [Medline: 11431231]

4. Kessler RC, Angermeyer M, Anthony JC, DE Graaf R, Demyttenaere K, Gasquet I, et al. Lifetime prevalence and age-of-onsetdistributions of mental disorders in the World Health Organization's World Mental Health Survey Initiative. World Psychiatry2007 Oct;6(3):168-176 [FREE Full text] [Medline: 18188442]

5. Kilbourne AM, Daugherty B, Pincus HA. What do general medical guidelines say about depression care? Depressiontreatment recommendations in general medical practice guidelines. Curr Opin Psychiatry 2007 Nov;20(6):626-631. [doi:10.1097/YCO.0b013e3282f0c4d3] [Medline: 17921767]



XSL•FORenderX


















6. Wittchen HU, Jacobi F, Rehm J, Gustavsson A, Svensson M, Jönsson B, et al. The size and burden of mental disorders andother disorders of the brain in Europe 2010. Eur Neuropsychopharmacol 2011 Sep;21(9):655-679. [doi:10.1016/j.euroneuro.2011.07.018] [Medline: 21896369]

7. Cuijpers P, van Straten A, Andersson G, van Oppen P. Psychotherapy for depression in adults: a meta-analysis of comparativeoutcome studies. J Consult Clin Psychol 2008 Dec;76(6):909-922. [doi: 10.1037/a0013075] [Medline: 19045960]

8. Bebbington P, Brugha T, Meltzer H, Jenkins R, Ceresa C, Farrell M, et al. Neurotic disorders and the receipt of psychiatrictreatment. Int Rev Psychiatry 2003;15(1-2):108-114. [doi: 10.1080/0954026021000046010] [Medline: 12745317]

9. Kessler RC, Berglund PA, Bruce ML, Koch JR, Laska EM, Leaf PJ, et al. The prevalence and correlates of untreated seriousmental illness. Health Serv Res 2001 Dec;36(6 Pt 1):987-1007 [FREE Full text] [Medline: 11775672]

10. Churchill R, Hunot V, Corney R, Knapp M, McGuire H, Tylee A, et al. A systematic review of controlled trials of theeffectiveness and cost-effectiveness of brief psychological treatments for depression. Health Technol Assess 2001;5(35):1-173[FREE Full text] [Medline: 12387733]

11. Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive-behavioral therapy: a review ofmeta-analyses. Clin Psychol Rev 2006 Jan;26(1):17-31. [doi: 10.1016/j.cpr.2005.07.003] [Medline: 16199119]

12. Ruwaard J, Schrieken B, Schrijver M, Broeksteeg J, Dekker J, Vermeulen H, et al. Standardized web-based cognitivebehavioural therapy of mild to moderate depression: a randomized controlled trial with a long-term follow-up. Cogn BehavTher 2009;38(4):206-221. [doi: 10.1080/16506070802408086] [Medline: 19221919]

13. Andersson G, Bergström J, Holländare F, Carlbring P, Kaldo V, Ekselius L. Internet-based self-help for depression:randomised controlled trial. Br J Psychiatry 2005 Nov;187:456-461 [FREE Full text] [doi: 10.1192/bjp.187.5.456] [Medline:16260822]

14. Berger T, Hämmerli K, Gubser N, Andersson G, Caspar F. Internet-based treatment of depression: a randomized controlledtrial comparing guided with unguided self-help. Cogn Behav Ther 2011;40(4):251-266. [doi: 10.1080/16506073.2011.616531][Medline: 22060248]

15. Johansson R, Sjöberg E, Sjögren M, Johnsson E, Carlbring P, Andersson T, et al. Tailored vs. standardized Internet-basedcognitive behavior therapy for depression and comorbid symptoms: a randomized controlled trial. PLoS One 2012;7(5):e36905[FREE Full text] [doi: 10.1371/journal.pone.0036905] [Medline: 22615841]

16. McCrone P, Knapp M, Proudfoot J, Ryden C, Cavanagh K, Shapiro DA, et al. Cost-effectiveness of computerisedcognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial. Br J Psychiatry 2004Jul;185:55-62 [FREE Full text] [Medline: 15231556]

17. Marks IM, Mataix-Cols D, Kenwright M, Cameron R, Hirsch S, Gega L. Pragmatic evaluation of computer-aided self-helpfor anxiety and depression. Br J Psychiatry 2003 Jul;183:57-65 [FREE Full text] [Medline: 12835245]

18. Marks IM, Kenwright M, McDonough M, Whittaker M, Mataix-Cols D. Saving clinicians' time by delegating routineaspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol Med 2004 Jan;34(1):9-17.[Medline: 14971623]

19. Titov N. Internet-delivered psychotherapy for depression in adults. Curr Opin Psychiatry 2011 Jan;24(1):18-23. [doi:10.1097/YCO.0b013e32833ed18f] [Medline: 20827199]

20. Christensen H, Griffiths KM, Korten AE, Brittliffe K, Groves C. A comparison of changes in anxiety and depressionsymptoms of spontaneous users and trial participants of a cognitive behavior therapy website. J Med Internet Res 2004 Dec22;6(4):e46 [FREE Full text] [doi: 10.2196/jmir.6.4.e46] [Medline: 15631970]

21. Cuijpers P, Donker T, Johansson R, Mohr DC, van Straten A, Andersson G. Self-guided psychological treatment fordepressive symptoms: a meta-analysis. PLoS One 2011;6(6):e21274 [FREE Full text] [doi: 10.1371/journal.pone.0021274][Medline: 21712998]

22. Meyer B, Berger T, Caspar F, Beevers CG, Andersson G, Weiss M. Effectiveness of a novel integrative online treatmentfor depression (Deprexis): randomized controlled trial. J Med Internet Res 2009;11(2):e15 [FREE Full text] [doi:10.2196/jmir.1151] [Medline: 19632969]

23. Clarke G, Kelleher C, Hornbrook M, Debar L, Dickerson J, Gullion C. Randomized effectiveness trial of an Internet, pureself-help, cognitive behavioral intervention for depressive symptoms in young adults. Cogn Behav Ther 2009;38(4):222-234[FREE Full text] [doi: 10.1080/16506070802675353] [Medline: 19440896]

24. Farrer L, Christensen H, Griffiths KM, Mackinnon A. Internet-based CBT for depression with and without telephonetracking in a national helpline: randomised controlled trial. PLoS One 2011;6(11):e28099 [FREE Full text] [doi:10.1371/journal.pone.0028099] [Medline: 22140514]

25. Lintvedt OK, Griffiths KM, Sørensen K, Østvik AR, Wang CE, Eisemann M, et al. Evaluating the effectiveness and efficacyof unguided Internet-based self-help intervention for the prevention of depression: a randomized controlled trial. ClinPsychol Psychother 2013;20(1):10-27. [doi: 10.1002/cpp.770] [Medline: 21887811]

26. Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms ofdepression and anxiety: a meta-analysis. Psychol Med 2007 Mar;37(3):319-328. [doi: 10.1017/S0033291706008944][Medline: 17112400]

27. Johansson R, Andersson G. Internet-based psychological treatments for depression. Expert Rev Neurother 2012Jul;12(7):861-9; quiz 870. [doi: 10.1586/ern.12.63] [Medline: 22853793]



XSL•FORenderX





















































28. Palmqvist B, Carlbring P, Andersson G. Internet-delivered treatments with or without therapist input: does the therapistfactor have implications for efficacy and cost? Expert Rev Pharmacoecon Outcomes Res 2007 Jun;7(3):291-297. [doi:10.1586/14737167.7.3.291] [Medline: 20528315]

29. Sethi S, Campbell AJ, Ellis LA. The use of computerized self-help packages to treat adolescent depression and anxiety.Journal of Technology in Human Services 2010 Aug 31;28(3):144-160. [doi: 10.1080/15228835.2010.508317]

30. Perini S, Titov N, Andrews G. Clinician-assisted Internet-based treatment is effective for depression: randomized controlledtrial. Aust N Z J Psychiatry 2009 Jun;43(6):571-578. [doi: 10.1080/00048670902873722] [Medline: 19440890]

31. Cuijpers P, Donker T, van Straten A, Li J, Andersson G. Is guided self-help as effective as face-to-face psychotherapy fordepression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med2010 Dec;40(12):1943-1957. [doi: 10.1017/S0033291710000772] [Medline: 20406528]

32. Groves C, Griffiths K, Christensen H. Working out MoodGYM: A User’s Guide. Canberra, Australia: Centre for MentalHealth Research; 2003.

33. Powell J, Hamborg T, Stallard N, Burls A, McSorley J, Bennett K, et al. Effectiveness of a web-based cognitive-behavioraltool to improve mental well-being in the general population: randomized controlled trial. J Med Internet Res 2013;15(1):e2[FREE Full text] [doi: 10.2196/jmir.2240] [Medline: 23302475]

34. Christensen H, Griffiths KM, Jorm AF. Delivering interventions for depression by using the Internet: randomised controlledtrial. BMJ 2004 Jan 31;328(7434):265 [FREE Full text] [doi: 10.1136/bmj.37945.566632.EE] [Medline: 14742346]

35. Calear AL, Christensen H, Mackinnon A, Griffiths KM, O'Kearney R. The YouthMood Project: a cluster randomizedcontrolled trial of an online cognitive behavioral program with adolescents. J Consult Clin Psychol 2009 Dec;77(6):1021-1032.[doi: 10.1037/a0017391] [Medline: 19968379]

36. Mackinnon A, Griffiths KM, Christensen H. Comparative randomised trial of online cognitive-behavioural therapy and aninformation website for depression: 12-month outcomes. Br J Psychiatry 2008 Feb;192(2):130-134 [FREE Full text] [doi:10.1192/bjp.bp.106.032078] [Medline: 18245031]

37. Hickie IB, Davenport TA, Luscombe GM, Moore M, Griffiths KM, Christensen H. Practitioner-supported delivery ofInternet-based cognitive behaviour therapy: evaluation of the feasibility of conducting a cluster randomised trial. Med JAust 2010 Jun 7;192(11 Suppl):S31-S35. [Medline: 20528705]

38. Wang PS, Berglund P, Kessler RC. Recent care of common mental disorders in the United States : prevalence andconformance with evidence-based recommendations. J Gen Intern Med 2000 May;15(5):284-292 [FREE Full text] [Medline:10840263]

39. Jorm AF. Mental health literacy. Public knowledge and beliefs about mental disorders. Br J Psychiatry 2000 Nov;177:396-401[FREE Full text] [Medline: 11059991]

40. Bebbington P, Meltzer H, Brugha T, Farrell M, Jenkins R, Ceresa C, et al. Unequal access and unmet need: neurotic disordersand the use of primary care services. Int Rev Psychiatry 2003;15(1-2):115-122. [doi: 10.1080/0954026021000046029][Medline: 12745318]

41. Helsedirektoratet. Nasjonale retningslinjer for diagnostisering og behandling av voksne med depresjon i primær- ogspesialisthelsetjenesten. 2009. URL: http://www.helsedirektoratet.no/publikasjoner/nasjonale-retningslinjer-for-diagnostisering-og-behandling-av-voksne-med-depresjon-i-primer--og-spesialisthelsetjenesten/Sider/default.aspx [accessed 2013-05-08] [WebCite Cache ID 6GStEoITt]

42. National Institute for Health and Clinical Excellence. Depression in adults: the treatment and management of depressionin adults. 2009. URL: http://www.nice.org.uk/nicemedia/live/12329/45888/45888.pdf [accessed 2013-05-08] [WebCiteCache ID 6GStYHltL]

43. Angermeyer MC, Matschinger H. Public attitude towards psychiatric treatment. Acta Psychiatr Scand 1996Nov;94(5):326-336. [Medline: 9124079]

44. Priest RG, Vize C, Roberts A, Roberts M, Tylee A. Lay people's attitudes to treatment of depression: results of opinionpoll for Defeat Depression Campaign just before its launch. BMJ 1996 Oct 5;313(7061):858-859 [FREE Full text] [Medline:8870574]

45. van Schaik DJ, Klijn AF, van Hout HP, van Marwijk HW, Beekman AT, de Haan M, et al. Patients' preferences in thetreatment of depressive disorder in primary care. Gen Hosp Psychiatry 2004;26(3):184-189. [doi:10.1016/j.genhosppsych.2003.12.001] [Medline: 15121346]

46. King M, Davidson O, Taylor F, Haines A, Sharp D, Turner R. Effectiveness of teaching general practitioners skills in briefcognitive behaviour therapy to treat patients with depression: randomised controlled trial. BMJ 2002 Apr20;324(7343):947-950 [FREE Full text] [Medline: 11964340]

47. Davidsen A. Experiences of carrying out talking therapy in general practice: a qualitative interview study. Patient EducCouns 2008 Aug;72(2):268-275. [doi: 10.1016/j.pec.2008.03.020] [Medline: 18472244]

48. Backenstrass M, Joest K, Rosemann T, Szecsenyi J. The care of patients with subthreshold depression in primary care: isit all that bad? A qualitative study on the views of general practitioners and patients. BMC Health Serv Res 2007;7:190[FREE Full text] [doi: 10.1186/1472-6963-7-190] [Medline: 18031573]

49. Wiebe E, Greiver M. Using cognitive behavioural therapy in practice: qualitative study of family physicians' experiences.Can Fam Physician 2005 Jul;51:992-993 [FREE Full text] [Medline: 16926952]



XSL•FORenderX

















































50. Telford R, Hutchinson A, Jones R, Rix S, Howe A. Obstacles to effective treatment of depression: a general practiceperspective. Fam Pract 2002 Feb;19(1):45-52 [FREE Full text] [Medline: 11818349]

51. Mykletun A, Knudsen AK, Tangen T, Overland S. General practitioners' opinions on how to improve treatment of mentaldisorders in primary health care. Interviews with one hundred Norwegian general practitioners. BMC Health Serv Res2010;10:35 [FREE Full text] [doi: 10.1186/1472-6963-10-35] [Medline: 20144205]

52. Blashki G, Parsons J, Morgan H, Hickie IB, Davenport TA. Providing psychological treatments in general practice. Rationaleand practicalities. Aust Fam Physician 2003 Aug;32(8):625-630. [Medline: 12973871]

53. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports ofWeb-based and mobile health interventions. J Med Internet Res 2011;13(4):e126 [FREE Full text] [doi: 10.2196/jmir.1923][Medline: 22209829]

54. Melville KM, Casey LM, Kavanagh DJ. Dropout from Internet-based treatment for psychological disorders. Br J ClinPsychol 2010 Nov;49(Pt 4):455-471. [doi: 10.1348/014466509X472138] [Medline: 19799804]

55. Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of thequantitative and qualitative evidence. Psychol Med 2009 May;39(5):705-712. [doi: 10.1017/S0033291708004224] [Medline:18812006]

56. Beck AT, Steer RA, Brown GK. BDI-II, Beck Depression Inventory: Manual. San Antonio, TX: Psychological Corp; 1996.57. Arnau RC, Meagher MW, Norris MP, Bramson R. Psychometric evaluation of the Beck Depression Inventory-II with

primary care medical patients. Health Psychol 2001 Mar;20(2):112-119. [Medline: 11315728]58. Dozois D, Dobson KS, Ahnberg JL. A psychometric evaluation of the Beck Depression Inventory-II. Psychological

Assessment 1998;10(2):83-89. [doi: 10.1037/1040-3590.10.2.83]59. Carlbring P, Brunt S, Bohman S, Austin D, Richards J, Öst LG, et al. Internet vs. paper and pencil administration of

questionnaires commonly used in panic/agoraphobia research. Computers in Human Behavior 2007 May;23(3):1421-1434.[doi: 10.1016/j.chb.2005.05.002]

60. Holländare F, Andersson G, Engström I. A comparison of psychometric properties between internet and paper versions oftwo depression instruments (BDI-II and MADRS-S) administered to clinic patients. J Med Internet Res 2010;12(5):e49[FREE Full text] [doi: 10.2196/jmir.1392] [Medline: 21169165]

61. Naus MJ, Philipp LM, Samsi M. From paper to pixels: A comparison of paper and computer formats in psychologicalassessment. Computers in Human Behavior 2009 Jan;25(1):1-7. [doi: 10.1016/j.chb.2008.05.012]

62. Schulenberg SE, Yutrzenka BA. Equivalence of computerized and conventional versions of the Beck Depression Inventory-II(BDI-II). Curr Psychol 2001 Sep;20(3):216-230. [doi: 10.1007/s12144-001-1008-1]

63. Beck AT, Steer RA. Beck Anxiety Inventory: Manual. San Antonio, TX: The Psychological Corporation; 1993.64. Fydrich T, Dowdall D, Chambless DL. Reliability and validity of the Beck Anxiety Inventory. Journal of Anxiety Disorders

1992 Jan;6(1):55-61. [doi: 10.1016/0887-6185(92)90026-4]65. Steer RA, Ranieri WF, Beck AT, Clark DA. Further evidence for the validity of the Beck Anxiety Inventory with psychiatric

outpatients. Journal of Anxiety Disorders 1993 Jul;7(3):195-205. [doi: 10.1016/0887-6185(93)90002-3]66. Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult

Clin Psychol 1988 Dec;56(6):893-897. [Medline: 3204199]67. Hedman E, Ljótsson B, Rück C, Furmark T, Carlbring P, Lindefors N, et al. Internet administration of self-report measures

commonly used in research on social anxiety disorder: A psychometric evaluation. Computers in Human Behavior 2010Jul;26(4):736-740. [doi: 10.1016/j.chb.2010.01.010]

68. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand 1983 Jun;67(6):361-370. [Medline:6880820]

69. Bjelland I, Dahl AA, Haug TT, Neckelmann D. The validity of the Hospital Anxiety and Depression Scale. An updatedliterature review. J Psychosom Res 2002 Feb;52(2):69-77. [Medline: 11832252]

70. Andersson G, Kaldo-Sandström V, Ström L, Strömgren T. Internet administration of the Hospital Anxiety and DepressionScale in a sample of tinnitus patients. J Psychosom Res 2003 Sep;55(3):259-262. [Medline: 12932800]

71. McCue P, Buchanan T, Martin CR. Screening for psychological distress using Internet administration of the HospitalAnxiety and Depression Scale (HADS) in individuals with chronic fatigue syndrome. Br J Clin Psychol 2006 Nov;45(Pt4):483-498. [doi: 10.1348/014466505X82379] [Medline: 17076959]

72. EuroQol Group. About EQ-5D. 2013. URL: http://www.euroqol.org/about-eq-5d.html [accessed 2013-05-08] [WebCiteCache ID 6GSukzvVz]

73. Dolan P. Modeling valuations for EuroQol health states. Med Care 1997 Nov;35(11):1095-1108. [Medline: 9366889]74. Lamers LM, Bouwmans CA, van Straten A, Donker MC, Hakkaart L. Comparison of EQ-5D and SF-6D utilities in mental

health patients. Health Econ 2006 Nov;15(11):1229-1236. [doi: 10.1002/hec.1125] [Medline: 16625671]75. Shervin N, Dorrwachter J, Bragdon CR, Shervin D, Zurakowski D, Malchau H. Comparison of paper and computer-based

questionnaire modes for measuring health outcomes in patients undergoing total hip arthroplasty. J Bone Joint Surg Am2011 Feb 2;93(3):285-293. [doi: 10.2106/JBJS.I.01104] [Medline: 21266642]

76. Bushnell DM, Reilly MC, Galani C, Martin ML, Ricci JF, Patrick DL, et al. Validation of electronic data capture of theIrritable Bowel Syndrome--Quality of Life Measure, the Work Productivity and Activity Impairment Questionnaire for



XSL•FORenderX









































Irritable Bowel Syndrome and the EuroQol. Value Health 2006;9(2):98-105. [doi: 10.1111/j.1524-4733.2006.00087.x][Medline: 16626413]

77. Diener E, Emmons RA, Larsen RJ, Griffin S. The Satisfaction With Life Scale. J Pers Assess 1985 Feb;49(1):71-75. [doi:10.1207/s15327752jpa4901_13] [Medline: 16367493]

78. Pavot W, Diener E. Review of the Satisfaction With Life Scale. Psychological Assessment 1993;5(2):164-172. [doi:10.1037/1040-3590.5.2.164]

79. Clench-Aas J, Nes RB, Dalgard OS, Aarø LE. Dimensionality and measurement invariance in the Satisfaction with LifeScale in Norway. Qual Life Res 2011 Oct;20(8):1307-1317 [FREE Full text] [doi: 10.1007/s11136-011-9859-x] [Medline:21308414]

80. Glaesmer H, Grande G, Braehler E, Roth M. The German Version of the Satisfaction With Life Scale (SWLS): psychometricproperties, validity, and population-based norms. European Journal of Psychological Assessment 2011 May 3;27(2):127-132.[doi: 10.1027/1015-5759/a000058]

81. Meyerson P, Tryon WW. Validating Internet research: a test of the psychometric equivalence of Internet and in-personsamples. Behav Res Methods Instrum Comput 2003 Nov;35(4):614-620. [Medline: 14748506]

82. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International NeuropsychiatricInterview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV andICD-10. J Clin Psychiatry 1998;59 Suppl 20:22-33;quiz 34. [Medline: 9881538]

83. Saunders JB, Aasland OG, Babor TF, de la Fuente JR, Grant M. Development of the Alcohol Use Disorders IdentificationTest (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption--II. Addiction1993 Jun;88(6):791-804. [Medline: 8329970]

84. Reinert DF, Allen JP. The alcohol use disorders identification test: an update of research findings. Alcohol Clin Exp Res2007 Feb;31(2):185-199. [doi: 10.1111/j.1530-0277.2006.00295.x] [Medline: 17250609]

85. Babor TF, Higgins-Biddle JC, Saunders JB, Monteiro MG. The Alcohol Use Disorders Identification Test: Guidelines forUse in Primary Care 2nd edition. 2001. URL: http://whqlibdoc.who.int/hq/2001/who_msd_msb_01.6a.pdf [accessed2013-05-10] [WebCite Cache ID 6GVRHvzpE]

86. Berman AH, Bergman H, Palmstierna T, Schlyter F. Evaluation of the Drug Use Disorders Identification Test (DUDIT)in criminal justice and detoxification settings and in a Swedish population sample. Eur Addict Res 2005;11(1):22-31. [doi:10.1159/000081413] [Medline: 15608468]

87. Cavanagh K, Shapiro DA, Van Den Berg S, Swain S, Barkham M, Proudfoot J. The acceptability of computer-aidedcognitive behavioural therapy: a pragmatic study. Cogn Behav Ther 2009;38(4):235-246. [doi: 10.1080/16506070802561256][Medline: 19306147]

88. Garratt A, Bjørngaard JH, Dahle KA, Bjertnaes OA, Saunes IS, Ruud T. The Psychiatric Out-Patient ExperiencesQuestionnaire (POPEQ): data quality, reliability and validity in patients attending 90 Norwegian clinics. Nord J Psychiatry2006;60(2):89-96. [doi: 10.1080/08039480600583464] [Medline: 16635926]

89. Nasjonalt kunnskapssenter for helsetjenesten. PasOpp: hva er erfaringene dine som bruker av poliklinikk i psykiatriskhelsevern?. 2007. URL: http://www.kunnskapssenteret.no/Sp%C3%B8rreskjemabanken/Spesialisthelsetjeneste+-+Psykisk+helsevern/Voksne%2C+psykisk+helsevern%2C+poliklinikk.9716.cms [accessed2013-05-10] [WebCite Cache ID 6GVRtgBlw]

90. Australian National University, Centre for Mental Health Research. The MoodGYM training program: mark III. 2008.URL: https://moodgym.anu.edu.au/welcome [accessed 2013-05-10] [WebCite Cache ID 6GVUeWUTc]

91. Christensen H, Griffiths KM, Mackinnon AJ, Brittliffe K. Online randomized controlled trial of brief and full cognitivebehaviour therapy for depression. Psychol Med 2006 Dec;36(12):1737-1746. [doi: 10.1017/S0033291706008695] [Medline:16938144]

92. Twisk J. Applied multilevel analysis: a practical guide. Cambridge, UK: Cambridge University Press; 2006.93. Silfvernagel K, Carlbring P, Kabo J, Edström S, Eriksson J, Månson L, et al. Individually tailored Internet-based treatment

for young adults and adults with panic attacks: randomized controlled trial. J Med Internet Res 2012;14(3):e65 [FREE Fulltext] [doi: 10.2196/jmir.1853] [Medline: 22732098]

94. Morris SB, DeShon RP. Combining effect size estimates in meta-analysis with repeated measures and independent-groupsdesigns. Psychol Methods 2002 Mar;7(1):105-125. [Medline: 11928886]

95. Thorsell J, Finnes A, Dahl J, Lundgren T, Gybrant M, Gordh T, et al. A comparative study of 2 manual-based self-helpinterventions, acceptance and commitment therapy and applied relaxation, for persons with chronic pain. Clin J Pain 2011Oct;27(8):716-723. [doi: 10.1097/AJP.0b013e318219a933] [Medline: 21540740]

96. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: L Erlbaum Associates; 1988.97. Seggar LB, Lambert MJ, Hansen NB. Assessing clinical significance: Application to the Beck Depression Inventory.

Behavior Therapy 2002 Mar;33(2):253-269. [doi: 10.1016/s0005-7894(02)80028-4]98. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research.

J Consult Clin Psychol 1991 Feb;59(1):12-19. [Medline: 2002127]



XSL•FORenderX









































99. Choi I, Zou J, Titov N, Dear BF, Li S, Johnston L, et al. Culturally attuned Internet treatment for depression amongstChinese Australians: a randomised controlled trial. J Affect Disord 2012 Feb;136(3):459-468. [doi: 10.1016/j.jad.2011.11.003][Medline: 22177742]

100. Vernmark K, Lenndin J, Bjärehed J, Carlsson M, Karlsson J, Oberg J, et al. Internet administered guided self-help versusindividualized e-mail therapy: A randomized trial of two versions of CBT for major depression. Behav Res Ther 2010May;48(5):368-376. [doi: 10.1016/j.brat.2010.01.005] [Medline: 20152960]

101. Barrett JE, Williams JW, Oxman TE, Frank E, Katon W, Sullivan M, et al. Treatment of dysthymia and minor depressionin primary care: a randomized trial in patients aged 18 to 59 years. J Fam Pract 2001 May;50(5):405-412. [Medline:11350703]

102. Hermens ML, van Hout HP, Terluin B, van der Windt DA, Beekman AT, van Dyck R, et al. The prognosis of minordepression in the general population: a systematic review. Gen Hosp Psychiatry 2004;26(6):453-462. [doi:10.1016/j.genhosppsych.2004.08.006] [Medline: 15567211]

103. Christensen H, Griffiths KM, Korten A. Web-based cognitive behavior therapy: analysis of site usage and changes indepression and anxiety scores. J Med Internet Res 2002;4(1):e3 [FREE Full text] [doi: 10.2196/jmir.4.1.e3] [Medline:11956035]

104. Goldberg D, Bridges K, Duncan-Jones P, Grayson D. Detecting anxiety and depression in general medical settings. BMJ1988 Oct 8;297(6653):897-899 [FREE Full text] [Medline: 3140969]

105. Bados A, Balaguer G, Saldaña C. The efficacy of cognitive-behavioral therapy and the problem of drop-out. J Clin Psychol2007 Jun;63(6):585-592. [doi: 10.1002/jclp.20368] [Medline: 17457848]

106. Watkins E, Williams R. The efficacy of cognitive behavioral therapy. In: Checkley S, editor. The Management of Depression.Oxford, UK: Blackwell Science; 1998:165-188.

107. Wierzbicki M, Pekarik G. A meta-analysis of psychotherapy dropout. Professional Psychology: Research and Practice1993;24(2):190-195. [doi: 10.1037/0735-7028.24.2.190]

108. Lamers F, van Oppen P, Comijs HC, Smit JH, Spinhoven P, van Balkom AJ, et al. Comorbidity patterns of anxiety anddepressive disorders in a large cohort study: the Netherlands Study of Depression and Anxiety (NESDA). J Clin Psychiatry2011 Mar;72(3):341-348. [doi: 10.4088/JCP.10m06176blu] [Medline: 21294994]

109. Jacobi F, Wittchen HU, Holting C, Höfler M, Pfister H, Müller N, et al. Prevalence, co-morbidity and correlates of mentaldisorders in the general population: results from the German Health Interview and Examination Survey (GHS). PsycholMed 2004 May;34(4):597-611. [doi: 10.1017/S0033291703001399] [Medline: 15099415]

110. van Boeijen CA, van Oppen P, van Balkom AJ, Visser S, Kempe PT, Blankenstein N, et al. Treatment of anxiety disordersin primary care practice: a randomised controlled trial. Br J Gen Pract 2005 Oct;55(519):763-769 [FREE Full text] [Medline:16212851]

111. Longwell BT, Truax P. The differential effects of weekly, monthly, and bimonthly administrations of the beck DepressionInventory-II: Psychometric properties and clinical implications. Behavior Therapy 2005 Jun;36(3):265-275 [FREE Fulltext] [doi: 10.1016/S0005-7894(05)80075-9]

112. Hatzenbuehler LC, Parpal M, Matthews L. Classifying college students as depressed or nondepressed using the BeckDepression Inventory: an empirical analysis. J Consult Clin Psychol 1983 Jun;51(3):360-366. [Medline: 6863694]

113. Haaga DA. Introduction to the special section on stepped care models in psychotherapy. J Consult Clin Psychol 2000Aug;68(4):547-548. [Medline: 10965628]

114. Farrand P, Confue P, Byng R, Shaw S. Guided self-help supported by paraprofessional mental health workers: an uncontrolledbefore--after cohort study. Health Soc Care Community 2009 Feb;17(1):9-17. [doi: 10.1111/j.1365-2524.2008.00792.x][Medline: 18564197]

115. Titov N, Andrews G, Davies M, McIntyre K, Robinson E, Solley K. Internet treatment for depression: a randomizedcontrolled trial comparing clinician vs. technician assistance. PLoS One 2010;5(6):e10939 [FREE Full text] [doi:10.1371/journal.pone.0010939] [Medline: 20544030]

AbbreviationsAUDIT: The Alcohol Use Disorder Identification TestANOVA: analysis of varianceANCOVA: analysis of covarianceANU: Australian National UniversityBAI: Beck Anxiety InventoryBDI-II: Beck Depression Inventory, second editionCBT: cognitive behavioral therapyDSM-IV: Diagnostic and Statistical Manual of Mental Disorders, 4th editionDUDIT: The Drug Use Disorder Identification TestEQ-5D: EuroQol Group 5-Dimension Self-Report QuestionnaireEQ Index: index score of the EQ-5D



XSL•FORenderX


































EQ VAS: visual analog scale of the EQ-5DGP: general practitionerHADS: Hospital Anxiety and Depression ScaleICD-10: International Classification of Diseases, 10th editionITT: intention-to-treatSWLS: Satisfaction with Life Scale

Edited by G Eysenbach; submitted 19.05.13; peer-reviewed by T Berger, T Donker; comments to author 10.06.13; revised versionreceived 03.07.13; accepted 12.07.13; published 28.07.13

Please cite as:Høifødt RS, Lillevoll KR, Griffiths KM, Wilsgaard T, Eisemann M, Waterloo K, Kolstrup NThe Clinical Effectiveness of Web-Based Cognitive Behavioral Therapy With Face-to-Face Therapist Support for Depressed PrimaryCare Patients: Randomized Controlled TrialJ Med Internet Res 2013;15(7):e153URL: http://www.jmir.org/2013/7/e153/ doi:10.2196/jmir.2714PMID:

©Ragnhild Sørensen Høifødt, Kjersti R Lillevoll, Kathleen M Griffiths, Tom Wilsgaard, Martin Eisemann, Knut Waterloo, NilsKolstrup. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 28.07.2013. This is an open-accessarticle distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/),which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in theJournal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publicationon http://www.jmir.org/, as well as this copyright and license information must be included.



XSL•FORenderX






The Clinical Effectiveness of Web-Based Cognitive Behavioral Therapy With Face-to-Face Therapist...

Documents

Transcript of The Clinical Effectiveness of Web-Based Cognitive Behavioral Therapy With Face-to-Face Therapist...