Houwing etal AAP2013

10
Accident Analysis and Prevention 52 (2013) 144–153 Contents lists available at SciVerse ScienceDirect Accident Analysis and Prevention j ourna l ho me pa ge: www.elsevier.com/locate/aap Random and systematic errors in case–control studies calculating the injury risk of driving under the influence of psychoactive substances Sjoerd Houwing a,, Marjan Hagenzieker a , René P.M. Mathijssen a , Sara-Ann Legrand b , Alain G. Verstraete b , Tove Hels c , Inger Marie Bernhoft c , Kirsten Wiese Simonsen d , Pirjo Lillsunde e , Donata Favretto f , Santo D. Ferrara f , Marija Caplinskiene g , Kris L.L. Movig h , Karel A. Brookhuis i a SWOV, Institute for Road Safety Research, Duindoorn 32, 2262 AR Leidschendam, The Netherlands b Ghent University, Department of Clinical Chemistry, Microbiology and Immunology, De Pintelaan 185, 9000 Ghent, Belgium c Department of Transport, Technical University of Denmark, Bygningstorvet 116B, 2800 Kgs. Lyngby, Denmark d Section of Forensic Chemistry, Department of Forensic Medicine, Copenhagen University, Frederik V’s Vej 11, DK-2100 Copenhagen, Denmark e Alcohol and Drug Analytics Unit, National Institute for Health and Welfare, P.O. Box 30, FI-00271, Finland f TFA-UNPD Università di Padova, Via Falloppio 50, 35100 Padova, Italy g VTMT State Forensic Medicine Service under the Ministry of Justice of the Republic of Lithuania, Didlaukio g. 86E, LT-08303 Vilnius, Lithuania h Department of Clinical Pharmacy, Medisch Spectrum Twente, Enschede, The Netherlands i University of Groningen, Faculty of Behavioral and Social Sciences, Grote Kruisstraat 2/1, 9712 TS, Groningen, The Netherlands a r t i c l e i n f o Article history: Received 29 June 2012 Received in revised form 18 December 2012 Accepted 21 December 2012 Keywords: Case–control studies Errors Psychoactive substances Alcohol Drugs a b s t r a c t Between 2006 and 2010, six population based case–control studies were conducted as part of the Euro- pean research-project DRUID (DRiving Under the Influence of Drugs, alcohol and medicines). The aim of these case–control studies was to calculate odds ratios indicating the relative risk of serious injury in car crashes. The calculated odds ratios in these studies showed large variations, despite the use of uniform guidelines for the study designs. The main objective of the present article is to provide insight into the presence of random and systematic errors in the six DRUID case–control studies. Relevant information was gathered from the DRUID-reports for eleven indicators for errors. The results showed that differences between the odds ratios in the DRUID case–control studies may indeed be (partially) explained by ran- dom and systematic errors. Selection bias and errors due to small sample sizes and cell counts were the most frequently observed errors in the six DRUID case–control studies. Therefore, it is recommended that epidemiological studies that assess the risk of psychoactive substances in traffic pay specific attention to avoid these potential sources of random and systematic errors. The list of indicators that was identified in this study is useful both as guidance for systematic reviews and meta-analyses and for future epidemi- ological studies in the field of driving under the influence to minimize sources of errors already at the start of the study. © 2013 Published by Elsevier Ltd. 1. Introduction Between 2006 and 2010 six population based case–control stud- ies were conducted during the European research-project DRUID (DRiving Under the Influence of Drugs, alcohol and medicines) in order to determine the risk of being seriously injured while driving with psychoactive substances (Hels et al., 2011). These case–control studies were performed in Belgium (BE), Denmark (DK), Finland (FI), Italy (IT), Lithuania (LT) and the Netherlands (NL). All six studies screened for the same 23 substances applying uniform analyti- cal cut-off levels. Cases were seriously injured drivers admitted to Corresponding author. E-mail address: [email protected] (S. Houwing). hospitals after a traffic crash and controls were randomly selected drivers from the general traffic. In epidemiological research, case–control studies are used to compare determinants (e.g. the presence of drugs) between injured and non-injured drivers. The main outcome measure of case–control studies is the odds ratio, which estimates relative risk (RR), since relative risk calculations cannot always be used (Schmidt and Kohlmann, 2008). This is due to the fact that there is no denom- inator for the incidence rates, since the controls are only a sample of the underlying denominator (or study base). Some case–control studies may have the possibility to calculate relative risk. How- ever, when choosing logistic regression as method for analysis, odds ratios will per definition be the outcome measure. An example of such a study is a case–control study from New Zealand by Connor et al. (2004). Although an odds ratio and a relative risk calculation 0001-4575/$ see front matter © 2013 Published by Elsevier Ltd. http://dx.doi.org/10.1016/j.aap.2012.12.034

Transcript of Houwing etal AAP2013

Ro

SADa

b

c

d

e

f

g

h

i

a

ARR1A

KCEPAD

1

i(ows(sc

0h

Accident Analysis and Prevention 52 (2013) 144– 153

Contents lists available at SciVerse ScienceDirect

Accident Analysis and Prevention

j ourna l ho me pa ge: www.elsev ier .com/ locate /aap

andom and systematic errors in case–control studies calculating the injury riskf driving under the influence of psychoactive substances

joerd Houwinga,∗, Marjan Hagenziekera, René P.M. Mathijssena, Sara-Ann Legrandb,lain G. Verstraeteb, Tove Helsc, Inger Marie Bernhoft c, Kirsten Wiese Simonsend, Pirjo Lillsundee,onata Favretto f, Santo D. Ferrara f, Marija Caplinskieneg, Kris L.L. Movigh, Karel A. Brookhuis i

SWOV, Institute for Road Safety Research, Duindoorn 32, 2262 AR Leidschendam, The NetherlandsGhent University, Department of Clinical Chemistry, Microbiology and Immunology, De Pintelaan 185, 9000 Ghent, BelgiumDepartment of Transport, Technical University of Denmark, Bygningstorvet 116B, 2800 Kgs. Lyngby, DenmarkSection of Forensic Chemistry, Department of Forensic Medicine, Copenhagen University, Frederik V’s Vej 11, DK-2100 Copenhagen, DenmarkAlcohol and Drug Analytics Unit, National Institute for Health and Welfare, P.O. Box 30, FI-00271, FinlandTFA-UNPD – Università di Padova, Via Falloppio 50, 35100 Padova, ItalyVTMT – State Forensic Medicine Service under the Ministry of Justice of the Republic of Lithuania, Didlaukio g. 86E, LT-08303 Vilnius, LithuaniaDepartment of Clinical Pharmacy, Medisch Spectrum Twente, Enschede, The NetherlandsUniversity of Groningen, Faculty of Behavioral and Social Sciences, Grote Kruisstraat 2/1, 9712 TS, Groningen, The Netherlands

r t i c l e i n f o

rticle history:eceived 29 June 2012eceived in revised form8 December 2012ccepted 21 December 2012

eywords:ase–control studiesrrorssychoactive substances

a b s t r a c t

Between 2006 and 2010, six population based case–control studies were conducted as part of the Euro-pean research-project DRUID (DRiving Under the Influence of Drugs, alcohol and medicines). The aim ofthese case–control studies was to calculate odds ratios indicating the relative risk of serious injury in carcrashes. The calculated odds ratios in these studies showed large variations, despite the use of uniformguidelines for the study designs. The main objective of the present article is to provide insight into thepresence of random and systematic errors in the six DRUID case–control studies. Relevant informationwas gathered from the DRUID-reports for eleven indicators for errors. The results showed that differencesbetween the odds ratios in the DRUID case–control studies may indeed be (partially) explained by ran-dom and systematic errors. Selection bias and errors due to small sample sizes and cell counts were the

lcoholrugs

most frequently observed errors in the six DRUID case–control studies. Therefore, it is recommended thatepidemiological studies that assess the risk of psychoactive substances in traffic pay specific attention toavoid these potential sources of random and systematic errors. The list of indicators that was identifiedin this study is useful both as guidance for systematic reviews and meta-analyses and for future epidemi-ological studies in the field of driving under the influence to minimize sources of errors already at thestart of the study.

© 2013 Published by Elsevier Ltd.

. Introduction

Between 2006 and 2010 six population based case–control stud-es were conducted during the European research-project DRUIDDRiving Under the Influence of Drugs, alcohol and medicines) inrder to determine the risk of being seriously injured while drivingith psychoactive substances (Hels et al., 2011). These case–control

tudies were performed in Belgium (BE), Denmark (DK), Finland

FI), Italy (IT), Lithuania (LT) and the Netherlands (NL). All six studiescreened for the same 23 substances applying uniform analyti-al cut-off levels. Cases were seriously injured drivers admitted to

∗ Corresponding author.E-mail address: [email protected] (S. Houwing).

001-4575/$ – see front matter © 2013 Published by Elsevier Ltd.ttp://dx.doi.org/10.1016/j.aap.2012.12.034

hospitals after a traffic crash and controls were randomly selecteddrivers from the general traffic.

In epidemiological research, case–control studies are usedto compare determinants (e.g. the presence of drugs) betweeninjured and non-injured drivers. The main outcome measure ofcase–control studies is the odds ratio, which estimates relative risk(RR), since relative risk calculations cannot always be used (Schmidtand Kohlmann, 2008). This is due to the fact that there is no denom-inator for the incidence rates, since the controls are only a sampleof the underlying denominator (or study base). Some case–controlstudies may have the possibility to calculate relative risk. How-

ever, when choosing logistic regression as method for analysis, oddsratios will per definition be the outcome measure. An example ofsuch a study is a case–control study from New Zealand by Connoret al. (2004). Although an odds ratio and a relative risk calculation

S. Houwing et al. / Accident Analysis and Prevention 52 (2013) 144– 153 145

F(n

aoiborrG

colcwwmI

udeB(2ecTc1csvIdt

vctresdtl

rei

ind

icat

ing

the

rela

tive

risk

of

bein

g

seri

ousl

y

inju

red

(MA

IS

≥2)

du

e

to

dri

vin

g

un

der

the

infl

uen

ce

of

psy

choa

ctiv

e su

bsta

nce

s

acco

rdin

g

to

cou

ntr

y,

adju

sted

for

age

and

gen

der

, an

d

95%

con

fid

ence

inte

rval

s

for

m

Hel

s

et

al.,

2011

);

na:

not

pos

sibl

e

to

calc

ula

te

du

e

to

low

cell

cou

nts

;

Inf:

infi

nit

y.

Bel

giu

mD

enm

ark

Fin

lan

dIt

aly

Lith

uan

iaN

eth

erla

nd

s

All

cou

ntr

ies

0.1–

0.5

g/L

1.03

0.49

–2.1

5

1.47

0.79

–2.7

4

6.55

0.81

–53.

25

0.56

0.29

–1.0

6

1.49

0.54

–4.1

3

1.58

0.49

–5.1

2

1.18

0.81

–1.7

3

0.5–

0.8

g/L

2.27

0.94

–5.4

9

5.66

2.5–

12.8

2

36.0

1

3.14

–413

.06

0.58

0.26

–1.2

9

3.69

0.91

–15.

02

9.4

2.89

–30.

61

3.64

2.31

–5.7

2

0.8–

1.2

g/L

13.2

3

5.61

–31.

21

14.3

2

4.68

–43.

87

55.0

7

2.74

–in

f.

1.53

0.76

–3.1

10.8

23.0

3–21

.22

31.3

7

11.3

4–86

.83

13.3

5

8.15

–21.

88

≥1.2

g/L

108.

6857

.5–2

05.4

3

296.

99

58.8

4–in

f.

128.

8438

.69–

429.

03

16.5

5

8.8–

31.1

5

11.4

26.1

4–21

.22

108.

0952

.45–

222.

75

62.7

944.

51–8

8.58

s amin

en

a49

.94

2.8–

891.

67

na

na

0.5

0.04

–6.8

8

8.87

1.84

–42.

86

8.35

3.91

–17.

83ec

gon

ine

na

na

na

3.24

0.85

–12.

38

na

12.2

3

2.86

–52.

34

3.7

1.6–

8.57

na

na

na

1.17

0.4–

3.4

na

na

3.3

1.4–

7.79

s

4.88

1.6–

14.8

4

2.17

0.61

–7.7

9

25.3

8

1.86

–345

.78

1.88

0.85

–4.1

7

na

0.29

0.04

–2.1

1

1.38

0.88

–2.1

7ia

tes

and

opio

ids

na

na

na

1.38

0.25

–7.6

2

na

na

2.47

0.5–

12.1

dru

gsaz

epin

es

and

Z-d

rugs

2.3

1.07

–4.9

4

4.37

2.18

–8.7

5

2.59

0.34

–19.

86

0.2

0.04

–1.0

1.02

0.36

–2.8

7

2.56

0.34

–19.

36

1.99

1.36

–2.9

1al

opio

ids

4.33

1.58

–11.

89

5.72

3.06

–10.

67

5.4

0.68

–42.

97

11.1

63.3

8–36

.88

na

5.96

0.73

–48.

84

9.06

6.4–

12.8

3io

ns

–dru

gs

58.1

627.

05–1

25.0

7

52.6

816.

01–1

73.3

5

148.

7

26.8

4–82

3.94

7.3

3.49

–15.

27

127.

32

4.22

–in

f.

12.5

5

4.76

–33.

12

28.8

218.

41–4

5.11

t

dru

gs

clas

ses

9.99

3.61

–27.

68

57.5

412.

66–2

61.5

3

45.8

6

7.92

–265

.38

2.29

1.12

–4.6

6

na

na

8.01

5.34

–12.

01

ig. 1. Calculation of odds ratios (OR) and relative risk (RR); derived from Hels et al.2011); a is the number of positive cases, c is the number of negative cases, b is theumber of positive controls, and d is the number of negative controls.

re not identical measures, the odds ratio can be used as an estimatef the relative risk, as long as the likelihood of a crash is small. Fig. 1llustrates that the odds ratio approximates the RR when the num-er of cases (a and c) is relatively small compared to the numberf controls (b and d). An odds ratio larger than 1 indicates a higherelative risk compared to a sober driver, while conversely an oddsatio smaller than 1 indicates a lower relative risk (Rothman andreenland, 1998).

Odds ratios were calculated for each of the six individualountries as well as for all six countries combined. The advantagef a pooled odds ratio was that the number of samples was mucharger so that odds ratios for less prevalent substances could bealculated. The odds ratios of the six DRUID case–control studies asell as for all countries together are presented in Table 1. Driversere considered seriously injured in case of a MAIS-score (Maxi-um Abbreviated Injury Scale) of 2 or higher (Garthe et al., 1999;

salberti et al., 2011).In general, the highest odds ratios were calculated for drivers

sing alcohol–drug combinations, drug–drug combinations and forrivers with high blood alcohol concentration (BAC) levels (Helst al., 2011). The risk posed by alcohol use increased at higherAC-levels, as has been found in previous case–control studiesBorkenstein et al., 1974; Compton et al., 2002; Krüger and Vollrath,004; Mathijssen and Houwing, 2005; Movig et al., 2004). How-ver, the results of the DRUID case–control studies show that thealculated odds ratios for alcohol varied widely between countries.he odds ratios for illicit drug use showed large variations betweenountries as well, with odds ratios differing by up to a factor of00. However, the confidence intervals were wide and for mostountries the number of positive samples for illicit drugs was toomall to calculate an odds ratio. The odds ratios for medicinal drugsaried less than those for illicit drugs and alcohol. Nonetheless, intaly the calculated odds ratio of 0.2 for benzodiazepines and Z-rugs (including zolpidem and zopiclone) was much lower thanhe odds ratio of 4.37 found in Denmark.

In sum, the results from these case–control studies showed largeariations in the relative risks for driving under the influence of psy-hoactive substances. The differences between the odds ratios ofhe case–control studies could reflect actual differences in relativeisk for driving under the influence in the different countries. How-ver, it is hard to believe that drivers in different countries showuch large differences for the relative risk of serious injury whileriving under the influence of psychoactive substances. Therefore,he observed differences between the odds ratios could also be ateast partially explained by different types of errors.

Depending on the source of the error they can be divided intoandom errors and systematic errors (Rothman, 1986). A randomrror can be described as sampling variability. This variabilitys visualized in case–control studies by confidence intervals. An Ta

ble

1O

dd

s

rati

os

(ad

apte

d

fro

Alc

ohol

Alc

ohol

Alc

ohol

Alc

ohol

Alc

ohol

Illi

cit

dru

gA

mp

het

Ben

zoyl

Coc

ain

eC

ann

abi

Illi

cit

opM

edic

inal

Ben

zod

iM

edic

inC

ombi

nat

Alc

ohol

Dif

fere

n

1 sis an

iitai

betFfoidfpabsdoigoqowcdaH

crtdasfipttscsdhao1odsttbp

(

ct

smn

46 S. Houwing et al. / Accident Analy

ncrease of the sample size leads to a decrease of the confidencenterval, producing results with higher precision and power. A sys-ematic error is often referred to as bias. Bias is generally describeds the difference between the observed value and the true value. Its, unlike random error, not affected by sample size.

In epidemiological literature different types categorisations ofias exist. In this study we used the classifications of Kleinbaumt al. (1982) and Rothman (1986) who distinguish three mainypes of bias: selection bias, information bias and confounding.irstly, selection bias can be described as a deflection that resultsrom the selection process of the study population. An examplef selection bias is detection bias, where there are differencesn detecting exposed and non-exposed participants. For instance,etection bias may occur when drivers are not selected at randomrom moving traffic, but mainly based on suspicion. As a result therevalence of psychoactive substances in traffic is overestimatednd the odds ratio will consequently be underestimated. Selectionias is regarded as the most common type of bias in case–controltudies (Schulz and Grimes, 2002). Secondly, information bias can beescribed as a deviation based on inaccuracy of the measurement,r classification, of study variables. An example of information biass misclassification bias where a subject is assigned to a wrongroup due to a measurement error of an instrument, or in casef an interview due to selective recall or misunderstanding theuestions. Finally, confounding arises when a result is distorted byne or more variables that are associated both with exposure andith the outcome (Rothman, 1986). The most commonly included

onfounding factors in case–control studies that assess the risk ofriving under the influence of psychoactive substances are gendernd age (Blows et al., 2004; Brault et al., 2004; Haworth et al., 1997;els et al., 2011; Mathijssen and Houwing, 2005; Mura et al., 2003).

Guidelines were formulated at the start of the DRUID surveysoncerning several design issues to ensure that the results wereepresentative and comparable (Assum et al., 2007). According tohese guidelines a cross-sectional roadside survey had to be con-ucted to determine the prevalence of psychoactive substancesmong the general driving population. The method of this roadsideurvey was based on a stratified multi-stage sampling design. In therst stage of this design study regions were selected in the partici-ating countries. These regions were meant to be representative forhe whole country with regard to substance use and traffic. Withinhese regions smaller research areas were selected in the secondtage, and within these areas, survey locations were selected, wherear drivers and drivers from vans aged 18 and above were randomlyelected from actual traffic. In each country, data were collecteduring several roadside survey sessions distributed over eight 6-

time periods covering all hours of the day on both weekdaysnd weekend days. The time periods were distributed into typef day (weekday–weekend day) and time of the day (04:00–09:59,0:00–15:59, 16:00–21:59, 22:00–03:59). Due to the preferencesf the police that had to be taken into account, it turned out that theistribution of the samples over the different time periods in theix countries could not always be representative for the distribu-ion of traffic volumes over the time periods. In order to adjust forhese differences, weight factors were calculated for each countryy dividing the national traffic proportion per time period by theroportion of sampled drivers for these time periods.

The case population of the consisted of seriously injured driversMAIS ≥ 2) of cars and small vans aged 18 and above.

A list of 23 core substances was selected together with analyticalut-off levels for whole blood and oral fluid. Oral fluid samples hado be collected by means of the Statsure Saliva Sampler device.

However, for practical, financial and/or legal reasons not alltudies were conducted according to these guidelines. Further-ore, some issues were not addressed in the guidelines, but could

onetheless have caused variation in the odds ratios (e.g. the

d Prevention 52 (2013) 144– 153

minimum number of samples and the maximum proportion of non-response). Therefore, several types of random and systematic errorscould have led to differences in the results between the six DRUIDcase–control studies.

The main objective of this study is to provide insight in thedifferent sources of errors that could explain the variance in theresults of the six DRUID case–control studies. The results of thisassessment may clarify the large inter-country variation that wasobserved between the odds ratios.

2. Methods

The method that was applied to explore the source of errors inthe DRUID case–control studies consisted of three steps: the firststep was to prepare a list of potential indicators for errors basedon the available data from the six case–control studies. Table 2provides an overview of these indicators for potential errors cat-egorized by type of error (random and systematic error) and bytype of bias (selection bias, information bias and confounding)according to the categorisation of Kleinbaum et al. (1982) andRothman (1986). This categorization is supported by Wacholderet al. (1992) who wrote two companion papers on issues involvedin selecting controls for case–control studies. They state that theselection of controls should be comparable to the selection of casesin three ways: the study base should be the same, confoundingfactors should be used to eliminate any distortion by other fac-tors, and the measurement errors should be comparable. Theseprinciples should reduce the three previously mentioned types ofbias in case–control studies: selection bias, information bias andconfounding. However, they also state that the effectiveness ofthese principles is constrained by the availability of resources andtime.

In the second step, information was gathered for each of thepotential indicators from the relevant DRUID-reports on prevalence(Houwing et al., 2011; Isalberti et al., 2011) and risk estimates (Helset al., 2011). These prevalence reports included the national reportsfor the six participating countries, which provided detailed infor-mation concerning the hospital studies and the roadside surveys.

In the third and final step, the collected information from theDRUID-reports was used to interpret the calculated odds ratios. Aquantitative assessment of bias was not possible, since this wouldrequire a large amount of detailed data on e.g. the use of psychoac-tive substances in the general population for each study region,which were not available. Therefore, in this study bias is only dis-cussed in qualitative terms based on the information derived fromthe reports on the prevalence and risk studies by searching fordeviations between the six studies on each of the indicators. Thelikelihood of bias was divided into two categories: no indication ofbias and indication of bias. In case of potential bias, an estimationof the size and direction of bias is provided when possible. Apartfrom the discussion on potential bias, random errors are discussedin terms of statistical power.

3. Results

Relevant information for each country was gathered from theindividual DRUID-reports and then categorized into the variousindicators for potential errors. Table 3 presents an overview of theresults of this analysis of potential errors.

3.1. Random errors

3.1.1. Sample sizeThe sample size is the number (n) of individual samples in a

study. The precision and thus statistical power of a study increases

S. Houwing et al. / Accident Analysis and Prevention 52 (2013) 144– 153 147

Table 2Indicators of potential errors.

Type of error Type of bias Indicator Short explanation

Random error Sample size Influences accuracyLow cell counts Low frequency in a cell leads to less accurate odds ratios

Systematic error Selection bias Geographic area covered by cases and controls Difference in area covered when sampling cases andcontrols may result in bias

Size of non-response Large non-response may result in larger biasAge and gender distribution of response and non-responsegroup

Differences may indicate non-response bias

Non-random sampling Over representation of specific groups may lead to biasInjury scale Differences in inclusion criteria may lead to incomparable

case populationsInformation bias Sampling method cases and controls Different sampling methods in cases and controls may lead

to biasAnalytical method Differences in sensitivity of the methods of analysis may

lead to biasTime between accident and sampling Differences in time between accident and sampling may

wtemotosb

DsawI

nt2bgNn

3

coarcooibvceyfIots

h

Confounding Confounding factors controlled for

ith the sample size. Statistical power represents the probabilityhat a test will find a significant difference if there is an actual differ-nce. If the power is low, the confidence intervals will increase anday result in fewer significant outcomes (Cohen, 1988). However,

dds ratios which are close to 1 are more sensitive to significancehan odds ratios that differ more from 1. Therefore the numberf cases and controls is used as an indicator for the power of thetudy, instead of the number of significant outcomes, which maye regarded as a more direct indicator.

The results for the assessment on the sample sizes of the sixRUID case–control studies are presented in Table 3. The sample

ize for the hospital cases varied between 54 for the Finnish studynd 839 for the Danish study. The sample size of the control sampleshich were collected at the roadside ranged between 1086 for the

talian study and 4822 for the Dutch study.Only the Belgian and the Danish study included relatively high

umber of samples in both the case (BE: 348 and DK: 839) andhe control populations (BE: 2949 and DK: 3002) (Houwing et al.,011; Isalberti et al., 2011). Despite the relatively small num-ers of samples in the control population, the Italian study has areater precision than the studies from Finland, Lithuania and theetherlands. This greater precision is a result of the relative largeumber of cases in the Italian case–control study (n = 676).

.1.2. Low cell countsAs described above, the sample size provides information con-

erning the study power. In addition, the distribution of the samplesver the four cells in a case–control study (see Fig. 1) provides valu-ble information on the accuracy of the outcomes as well. In theeport on risk estimates (Hels et al., 2011) the issue of low cellounts was explored by an alternative method to calculate overalldds ratios. In this alternative method the lowest cell count for eachdds ratio calculation was both decreased and increased by 1 tonvestigate the effect of a small change of the data. If the differenceetween the odds ratio of the decreased value and the increasedalue was larger than a factor 2, the study was regarded as too sus-eptible to bias due to low cell counts and therefore not sufficient tonable calculation of the overall odds ratios. The results of the anal-sis on low cell counts (Hels et al., 2011) showed low cell countsor all substances in the Finnish case–control study. The Lithuanian,talian and Dutch study each only had sufficient cell counts for threef the substance groups. In Denmark and Belgium the results of

he case–control study seemed to have the highest accuracy withufficient cell counts for five and six substance groups, respectively.

Therefore, the Belgian and Danish studies were considered toave the highest study power. Both studies were also the only

lead to biasControl for different confounding factors may lead to bias

studies that were regarded as having a sufficiently large samplesize of both cases and controls (see Section 3.1.1 on the samplesize).

3.2. Systematic errors

3.2.1. Geographic area covered by cases and controlsOne of the basic principles of a population based case–control

study is that both the case and control population should be drawnfrom the same study population (Rothman and Greenland, 1998). Ifthe use of psychoactive substances varies according to region, andthe cases are not collected from the same region as the controls,then the odds ratio calculations may not be valid.

The results of the analysis on the area covered by the casesand controls showed that the only two countries in which thecase–control study did not cover the same area for both cases andcontrols were Denmark and the Netherlands. Since there were nodifferences in the age and gender distributions between the hos-pital and non-hospital regions in both countries, a decision wasmade to include all the results from the Dutch and Danish roadsidestudy in the odds ratio calculations (Hels et al., 2011). However,a significantly lower prevalence was found for the total substanceuse in the Dutch regions that did not include a hospital providingcase samples. No significant differences were found for any of theseparate substance groups though. In Denmark no differences werefound for substance use in the additional calculations of the controlsamples. Therefore, we have concluded that the effect of selectionbias related to the areas covered by the cases and controls is notlikely to be of influence in Denmark and the Netherlands.

In Belgium the inclusion of hospital samples seemed to vary overthe study regions: the share of injured drivers in Flanders was over-represented (81% of the cases and 57% of the controls), and the sharein Wallonia (16% of the cases and 39% of the controls) was under-represented (Houwing et al., 2011; Isalberti et al., 2011). However,additional analysis of the prevalence results in traffic by region didnot show any significant differences between the substance use inFlanders and Wallonia. Therefore, the effect of bias due to selectionbias in the case samples in Belgium is expected to be very small. Forall other studies no selection bias was observed in the case samples.

3.2.2. Size of non-responseA large share of non-response in a study increases the likelihood

of a selection bias. This selection bias may lead to an underestima-tion of the share of illicit drug users, since drivers who were positivefor illicit drugs can be assumed to be less likely to participate vol-untarily because of the risk of being detected positive for drugs in

148 S. Houwing et al. / Accident Analysis and Prevention 52 (2013) 144– 153

Table 3Summary of results of the assessment of potential errors; see text for further explanation: AMP, amphetamines; BEN, benzoylecgonine, BZD, benzodiazepines, COC, cocaine,OPI, illicit opiates, ALC, alcohol, OF, oral fluid.

Indicator Belgium (BE) Denmark (DK) Finland (FI) Italy (IT) Lithuania (LT) Netherlands (NL)

Sample size N controls 2949N cases 348

N controls 3002N cases 839

N controls 2706N cases 54

N controls 1086N cases 676

N controls 1264N cases 385

N controls 4822N cases188

Low cell counts Sufficient cellcounts forcalculating OR’s forall substancesexcept AMP, BEN,COC and OPI

Sufficient cellcounts forcalculating OR’s forall substancesexcept AMP, BEN,COC, OPI, anddrug–drugcombinations

Insufficient cellcounts forcalculating OR’s forall substances

Sufficient cellcounts forcalculating OR’s forBEN, THC anddrug–drugcombinations

Sufficient cellcounts forcalculating OR’s forALC and BZD andZDR

Sufficient cellcounts forcalculating OR’s forALC, BEN andalcohol–drugscombinations

Geographic areacovered by cases andcontrols

Cases and controlscover the samearea, although thedistribution of thecases over thehospitals was notrepresentative

Cases and controlsdo not cover thesame area.However, nosignificantdifference in age,gender andsubstance use wasobserved betweenhospital andnon-hospitalregions

Cases and controlscover the samearea

Cases and controlscover the samearea

Cases and controlscover the samearea

Cases and controlsdo not cover thesame area. Nosignificantdifference in ageand gender wasobserved betweenregions, but totalsubstance usediffered

Size of non-response Controls 52%Cases 5.4%

Controls 5%Cases 5%

Controls 48%Cases 8.5%

Controls 0%Cases not observed

Controls 24%Cases not observed

Controls 5%Cases not known

Age and genderdistribution ofresponse andnon-response group

Controls:Overrepresentationof male driversaged 25–34 incontrol population.BAC distributionwas the samebetween responseand non-responsegroupCases: Nosignificantdifferences

Controls: Nosignificantdifferencesreported for ageand gender,although maledrivers 18–24 andfemale drivers35–49 weresomewhat overrepresentedNo informationavailable for cases

Controls: Noinformationavailable althougha higher prevalenceof substances wasreported in thenon-responsegroup based onpolice records.No informationavailable for cases

No non responseobserved in bothcases and controls

Controls: Femaledrivers, mainlybetween 18 and 34years, were overrepresented amongnon-respondentsNo informationavailable for cases

Controls: Malesaged 25–34 wereover representedand females aged35+ and males 50+were underrepresentedNo informationavailable for cases

Non-random sampling Not observed Not observed Not observed Part of the driverswere preselectedbased on clinicalsigns of alcoholimpairment

Not observed Not observed

Inclusion criteria casesand controls

MAIS2+ Danish Traumascore 2+

MAIS2+ Injury prognosis≥20 days

MAIS2+ MAIS2+

Sampling method casesand controls

Controls: Blood/OF(Statsure)Cases: Blood

Controls: OF(Statsure), for 94%of the controlsethanol byconversion breathalcohol into bloodalcoholCases: Blood

Controls: OF(Statsure), ethanolby conversionbreath alcohol intoblood alcoholCases: Blood

Controls Blood/OF(Statsure), ethanolby conversionbreath alcohol intoblood alcoholCases: Blood

Controls: BloodCases: Blood

Controls: Blood/OF(spit tubes),ethanol byconversion breathalcohol into bloodalcoholCases: Blood

Analytical method Extraction: Solidphase andliquid–liquid forTHCchromatographyandDetection:UPLC–MS/MSGC–MS for THC

Extraction: Solidphase andliquid-liquid forTHCchromatographyandDetection:UPLC–MS/MS

Extraction: Solidphase andliquid–liquidchromatographyandDetection: GC–MS

Extraction: Solidphase andliquid–liquid forTHCchromatographyandDetection:HPLC–MS/MSGC–MS for THC

Extraction: Solidphase andliquid–liquidchromatographyandDetection: GC–MS

Extraction:Liquid–liquidchromatographyandDetection:UPLC–MS/MS

Time between accidentand sampling

Mean 1.33 hMedian 1.20 h

Mean 1.05 hMedian 1.00 h

Mean 1.50 hMedian 1.83 h

Mean 1.42 hMedian 1.00 h

Mean 0.78 hMedian 0.75 h

Mean 1.21 hMedian 1.08 h

Confounding factorscontrolled for

Age and gender,control populationweighted for thevolume of thetraffic flow over 8different timeperiods

Age and gender,control populationweighted for thevolume of thetraffic flow over 8different timeperiods

Age and gender,control populationweighted for thevolume of thetraffic flow over 8different timeperiods

Age and gender,control populationweighted for thevolume of thetraffic flow over 8different timeperiods

Age and gender,control populationweighted for thevolume of thetraffic flow over 8different timeperiods

Age and gender,control populationweighted for thevolume of thetraffic flow over 8different timeperiods

sis an

tcpetln

vwnrtm

3g

rgctaroctdm(

iowawnpdtapmfpwsin

rrDbiotprs

3

sts

S. Houwing et al. / Accident Analy

he vicinity of the police who were present at the scene and tookare of stopping the drivers. The share of non-response in the hos-ital studies, varied between 0 (not observed) and 8.5% (Isalbertit al., 2011), which can be considered as relatively low. Based onhe information on the size of the non-response in the case popu-ation as shown in Table 3, it can be concluded that the presence ofon-response bias in the case population is likely to be small.

The non-response rates in the roadside surveys showed largerariations with a range of between 0 and 52%. In Italy, non-responseas non-existent since participation was mandatory. In Lithua-ia, Belgium and Finland the proportion of non-respondents at theoadside was very high at 25%, 48% and 52%, respectively. Based onhis information, we assess that there was likely to be an overesti-

ation of the odds ratios for illicit drugs in these three countries.

.2.3. Age and gender distribution of response and non-responseroup

Age and gender differences between the response and non-esponse groups may indicate selection bias, since both age andender can be associated with different patterns in drug use byar drivers (Hels et al., 2011). Results from the roadside surveys inhe DRUID project indicate that illicit drug use is more prevalentmong young male drivers, whereas the use of medicinal drugs iselatively more frequently detected in female drivers aged 35 andlder (Houwing et al., 2011). The only hospital study (cases) whichollected data from the non-response group was the Belgian hospi-al study. No significant differences were found for age and genderistribution in this study. For the other hospital studies no infor-ation was available (DK and FI) or no non-response was observed

IT, LT and NL).The results from the roadside surveys (controls) showed that

n Belgium and the Netherlands male drivers, aged 25–34, werever represented in the non-response group, in Denmark thereas an over representation of young male drivers, aged 18–24,

nd female drivers aged 30–49 and in Lithuania female driversere generally over represented. In the Italian roadside survey noon-response group was observed at all, since refusal to partici-ate led to a large penalty by the police. In Finland and Belgiumrivers were first stopped and breath tested by the police, beforehey were asked to participate in the study. In these two countries

large share of drivers already declined to participate when theolice asked them to drive to the researchers. In Finland, no infor-ation was recorded from these drivers on age, gender and reason

or refusal. But from police records information was available onositive substance detections in those non-participating drivershom the police suspected of driving under the influence of sub-

tances other than alcohol (Houwing et al., 2011). This informationndicated that substance use was more frequently detected amongon-respondents.

In summary, it can be concluded that differences between theesponse and non-response group may have led to be an under rep-esentation of illicit drug use by car drivers in Belgium and Finland.ifferences were detected in Denmark and the Netherlands as well,ut since the total non-response rate was relatively low the effect

s likely to be small. In Lithuania, it may be assumed that due to thever representation of female drivers in the non-response grouphe prevalence of illicit drugs in traffic is overestimated whereas therevalence of medicinal drugs is underestimated. In Italy no non-esponse was observed in either the hospital study or the roadsideurvey.

.2.4. Non-random sampling

One of the objectives of the hospital studies and roadside

urveys was to gather information on substance use for a represen-ative sample of drivers in order to calculate odds ratios. Therefore,ampling needed to be carried out on a random basis. The process

d Prevention 52 (2013) 144– 153 149

of random sampling can be endangered by selection bias. Detectionbias is a form of selection bias, which can occur due to oversam-pling of certain sub-populations. The presence of detection bias ishard to reveal. However, in the DRUID report on the case–controlstudies (Hels et al., 2011) it was noted, based on personal com-munication with the Italian researchers, that in the Italian roadsidesurvey drivers who showed clinical signs of alcohol impairment hadpartially been preselected. Therefore, alcohol use in Italian trafficis expected to be overestimated, causing an underestimation of theodds ratios for alcohol use. This detection bias may also have beenpresent for users of psychoactive substances other than alcohol, inthe event that they showed signs of impairment as well. We assessthat the Italian odds ratios for alcohol and other psychoactive sub-stances were likely to be underestimated. For all other countries noindication was found for bias due to non-random sampling.

3.2.5. Inclusion criteria for cases and controlsThe inclusion criteria for the roadside surveys were the same for

all countries: drivers of cars and small vans, during all times of theday and all days of the week (Houwing et al., 2011). For the hospitalstudies the inclusion criteria were seriously injured drivers, aged 18years and older, driving a car or small van, with a MAIS score (Maxi-mum Abbreviated Injury Scale) of 2 or higher (Isalberti et al., 2011).Furthermore, the time between the accident and sampling had tobe less than 3 h. In Denmark and Italy a national injury scale wasapplied instead of the MAIS-scale. In Denmark this scale was basedon injury severity and in Italy injured drivers were included if theprognosis of recovery from injury was 20 days or longer (Isalbertiet al., 2011). Based on the national study of Danish hospital resultsin the DRUID-report on risk estimates (Hels et al., 2011), no largedifferences are expected for the Danish study due to the differencein the inclusion criteria for injured drivers. The potential effect ofthe divergent Italian inclusion criteria is hard to estimate since it isbased on the number of days of care that is needed and not on theseverity of the injury.

3.2.6. Sampling method cases and controlsThe theoretically most appropriate method for comparison of

cases and controls is by comparing blood samples from injureddrivers with blood samples from non-injured drivers (Walsh et al.,2008). Blood samples were collected from all injured drivers in thehospital studies. However, for the roadside surveys blood sampleswere only collected in Lithuania (all samples), the Netherlands (78%of all samples), Belgium (93%) and Italy (73%). All other collectedsamples were oral fluid. Therefore, equivalent cut-offs betweenblood and oral fluid concentrations were developed in order to beable to compare blood and oral fluid concentrations (Verstraeteet al., 2011).

Another issue concerning sampling of body fluids was the differ-ence in collection method for oral fluid samples. In the Netherlandsoral fluid samples were collected by using polyethylene spit tubes,whereas in all the other countries oral fluid samples were collectedby using a Statsure© oral fluid collection device with non-acidicstimulation. The guidelines prescribed the use the Statsure© devicefor collecting oral fluid (Assum et al., 2007). However, the Dutchroadside survey had already started by the time this decision wasmade and in order to keep the same oral fluid collection procedureduring the whole study the Dutch researchers decided to continueworking with the spit tubes.

The equivalent cut-offs that were used in the DRUID-studieswere based on concentrations for the Statsure© collection device.A recent study (Houwing et al., 2012) shows that THC concentra-

tions in oral fluid collected using spit tubes are in general almost sixtimes higher than THC concentrations collected by a device withoutacidic stimulation. Results for codeine sampling by different oralfluid collection methods showed a factor ranging from 1.3 to 2.0

1 sis an

(tw

flbtbtD

Nh

bhfatcdss

c1patofL

t

3

bfoecus

rqem

3

ssabiwaiasvcd

50 S. Houwing et al. / Accident Analy

O’Neal et al., 2000). In the DRUID case–control studies, however,he same equivalent cut-offs were applied for samples collectedith spit tubes as for those with the Statsure© device.

It can be concluded that the potential bias by using blood-oraluid comparisons instead of blood-blood comparisons is likely toe absent for all countries except for the Netherlands where 22% ofhe roadside samples were analyzed in oral fluid. Therefore, it cane assumed that the difference in the sampling method betweenhe cases and controls has led to a (small) overestimation of theutch prevalence results in the controls.

In four of the six roadside surveys (Denmark, Finland, Italy, theetherlands), ethanol (alcohol) concentrations were measured byandheld breathalyzers of the police.

In Belgium, BAC was measured from both oral fluid and wholelood. The estimated blood ethanol concentration was a factor 1.22igher than the measured ethanol concentration in oral fluid. This

actor was based on the average conversion factor between bloodnd oral fluid that was calculated from the Belgian DRUID results ofhose drivers from whom both blood and oral fluid samples wereollected (Verstraete et al., unpublished observations). In Denmark,ue to missing data of the police, ethanol concentrations were mea-ured in oral fluid in 6% of the drivers at the roadside by using theame factor of 1.22 as was used in the Belgian study.

In the Netherlands, the conversion factor of breath alcohol con-entrations into blood alcohol concentrations in percentages is:23 (Mathijssen and Twisk, 2001). However, in the other Euro-ean countries that were involved in the DRUID roadside surveys

higher conversion factor of 1:21 is used (Melethil, 2011). In ordero be able to compare the Dutch alcohol results with the results forther EU countries, all blood alcohol concentrations (BAC) resultsrom the Netherlands were multiplied by a factor 1.095 (23/21). Inithuania, ethanol concentrations were directly measured in blood.

Based on the conversion factors that were applied, no substan-ial bias was expected for alcohol measurements.

.2.7. Method of toxicological analysisDifferences in accuracy of the analytical methods may lead to

iased results as well. Information on the methods was availableor the extraction, separation and detection methods. Extractionf the samples was based either on liquid-liquid or solid phasextraction, chromatographic separation was performed by gashromatography or liquid chromatography high performance orltra performance. Detection of substances was based on masspectrometry or nitrogen/phosphorus detection.

Four separate rounds of proficiency testing were performed. Theesults of the proficiency testing show that both qualitative anduantitative performance improved during the testing program (Pilt al., 2010). Therefore, bias due to differences in the analyticalethods is not likely.

.2.8. Time between accident and samplingBased on the information in the databases from the six hospital

tudies it can be concluded that time between accident and bloodampling varied between 0 min and 3 h. The time span betweenccident and sampling could cause an underestimation of the num-er of case samples (and concentration of substances therein). This

s due to the process of metabolism, since psychoactive substancesith a short half-life, such as heroin, could already be metabolized

fter only 3 h. For all countries the mean and median times, shownn Table 3, were between 1 and 2 h, except for Lithuania that shows

shorter time period. The recorded time between accident and

ampling in Lithuania had a timeframe with both mean and medianalues of approximately 45 min. Since the differences between theountries are relatively small and the maximum time between acci-ent and sampling was set at 3 h, we assess that the presence of bias

d Prevention 52 (2013) 144– 153

due to differences in time between accident and sampling is of thesame order for all countries.

3.2.9. Confounding factorsDeconfounding is, like the presence of an identical study base for

both cases and controls, one of the main principles of comparativecase and control selections (Wacholder et al., 1992). Confoundingfactors are variables that co-vary both with substance use and crashrisk. Taking into account different types of confounding factors cancause variation between the results of case–control studies. In theDRUID project all the case–control studies’ results were calculatedby the Technical University of Denmark (DTU), ensuring a uni-form method of statistical analysis. The results of all countries werebased on a similar set of variables (age and gender). Furthermore,the data from all six roadside surveys were weighted for the vol-ume of traffic flow in the different time periods. There are probablyother confounding factors that were not detected in these studies.Age and gender were included as confounding variables in all cal-culations and since the data were adjusted for differences in trafficvolume between time periods, and therefore we assume that thepotential effect of confounding has been reduced. However, sinceconfounding has only been eliminated for those factors that we haddata available for, any influence of confounding on the estimatedodds ratios cannot be ruled out.

4. Discussion and conclusions

Based on the results of the assessment of potential bias due torandom and systematic errors it can be concluded that each of thenational case–control studies showed indications of potential bias.

The assessment for the Belgian case–control indicated a poten-tial overestimation of the odds ratio for cannabis due to thelikelihood of non-response bias in the control sample, in combi-nation with a large proportion of non-respondents. Relatively highodds ratios for illicit drugs were observed which supports the like-lihood of the presence of at least some non-response bias in theroadside survey. For all other substances the odds ratios were rel-atively close to the overall odds ratio.

In Denmark, a small overestimation of the odds ratio for illicitdrugs was possible due to the likelihood of a non-response biasin the control sample. However, the effect of the potential non-response bias was probably small, because of the low non-responserate (5%). For alcohol and medicinal drugs the presence of bias wasnot likely.

In Finland, the odds ratio for illicit drugs and alcohol was likely tobe overestimated due to non-response bias in the control sample incombination with a large share of non-respondents. These expec-tations were supported by the results of the comparison with theoverall odds ratios for alcohol and illicit drugs in Table 1, whichshows relatively high odds ratios for Finland. The Finnish resultswere expected to be less precise because of the small number ofcases (n = 54). This has resulted in relatively large confidence inter-vals.

In Italy, the odds ratios for alcohol were likely to be underesti-mated due to a selection bias in the control population. There mayhave been an underestimation for the other substances as well,since the police tended to include drivers with clinical signs ofimpairment. These findings were supported by the results of Table 1in which generally the odds ratios for the Italian study were con-siderably lower than those of the other countries. A very important

principle of a case–control study is that selection of participants hasto be independent of exposure. Since this principle was not (fully)met in the Italian study, a proper interpretation of the Italian resultswas not possible.

sis an

lobt

dtpitwmaapo

m(a

theilf(rdstr

nshatptne

etnfasiarers

fdiptTfu

S. Houwing et al. / Accident Analy

In Lithuania, the odds ratios for alcohol and illicit drugs wereikely to be underestimated due to the large under representationf female drivers in combination with the large (24%) non-responseias. Furthermore, the precision of the study was relatively low dueo the small number of control samples.

In the Netherlands, an overestimation of the odds ratio for illicitrugs was possible due to the likelihood of a non-response bias inhe control sample and due to selection bias resulting from lowerrevalence of substance use in traffic in regions that were not

ncluded in the case study. The non-response rate was very lowhough and therefore, the effect of the potential non-response biasas likely to be small. On the other hand, a potential underesti-ation of the odds ratios can be noted for all substances, except

lcohol, due to the use of too low cut-offs in the control sample as result of the different method of oral fluid sampling. Finally, therecision of the study was relatively low due to the small numberf case samples.

Taking the results of all countries together, we conclude that theost commonly found errors seem to be caused by selection bias

systematic errors) and were due to relatively small sample sizesnd low cell counts for individual substances (random errors).

The present analysis of potential bias due to random and sys-ematic errors is qualitative. Therefore, it cannot be determinedere to what extent these errors would have explained the differ-nces in odds ratio between the countries. First of all, the confidencentervals of the odds ratios were very large which means that evenarge differences, such as the difference between the odds ratiosor cannabis in the Netherlands (OR 0.29, CI 0.04–2.11) and FinlandOR 25.38, CI 1.86–345.78), might (theoretically) be considered aesult arising from the different sample sizes. Furthermore, someifferences between the odds ratios of the DRUID case–controltudies may arise from unknown confounding factors that are coun-ry related as was expressed by Hargutt et al. (2011) (e.g. climate,oad conditions, and density of traffic).

The results of the assessment include a comparison of eachational case–control study with the other DRUID case–controltudies. This method was chosen for practical reasons. It mightave been theoretically more sound to use the national study thatdhered the best to the proposed study design, as a reference studyo which the results of the other national studies can be com-ared. However, since no quantitative information is available onhe effect of study errors it is difficult to determine which of theational studies would suffer the least from random and systematicrrors.

If we assume that the Danish study would provide the beststimates (as our assessment shows that this study suffers fromhe least sources of bias) additional issues arise. Due to the lowumber of positive samples, no odds ratios were calculated for the

ollowing substances: benzoylecgonine, cocaine and illicit opiatesnd opioids. This means that any results of other studies for theseubstances cannot be used for comparison. Furthermore, the Dan-sh odds ratio for amphetamines is 50 and the odds ratio for alcoholt a BAC of 1.2 g/L or higher is almost 300. These odds ratios seemelatively high, which may be caused by random and systematicrrors in the Danish study. Therefore, using these odds ratios as aeference for the odds ratios from the other five DRUID case–controltudies may lead to the wrong conclusions.

It is clear from the results of this study that the presence of uni-orm guidelines was not sufficient in excluding differences in theesign and protocol of the six national DRUID case–control stud-

es. Deviations from the guidelines such as those mentioned in theresent study were caused by practical, financial and legal limi-

ations. Such limitations are difficult to overcome for researchers.herefore, it would be utopic to expect that future studies will beully comparable with each other. Previous review studies on drugse in traffic that included case–control studies have discussed the

d Prevention 52 (2013) 144– 153 151

issue of comparability between different study types. Only twostudies were found in literature that have compared the calculatedodds ratios from a case–control study with the odds ratios that werecalculated by a case-crossover design (Brault et al., 2004; Raveraet al., 2011). The effects of errors on the outcomes of case–controlstudies are also discussed in literature. Connor et al. (2004) men-tioned the effect of errors on the comparability of case–controlstudies that studied the effect of alcohol use among drivers, as wasdone more recently by Krüger et al. (2008). Houwing et al. (2009)formulated a list of comparability indicators for case–control stud-ies assessing the risk of driving under the influence of psychoactivesubstances, including a list of study design items, which could leadto errors. Another review study on drugs and driving (OECD, 2010)listed methodological problems such as issues concerning bodyfluid samples, the elapsed time between accident and sampling,and the problem of determining the contribution of the substanceto the accident as opposed to the possible contribution of con-founding factors. The report concludes that the inconsistency ofresults of epidemiological studies on cannabis use is at least par-tially attributable to differences in study design and approach.Although these studies show that the presence of errors has beenmentioned in literature, none of the previous studies was able toprovide more insight into the degree that variation was caused bystudy errors. This is due to the fact that before the start of the Euro-pean DRUID-project there were no case–control studies conductedwith comparable study designs that estimated the risk of drivingunder the influence of psychoactive substances other than alco-hol. In the present study, the only major difference in study designbetween the six DRUID case–control studies was the use of twodifferent types of body fluids for analyzing substance use in thecontrol population. If we assume that the use of equivalent cut-offsadjusted for the larger part of differences in substance concentra-tions between both matrices, we can conclude that the variation inodds ratios caused by errors is in particular large for alcohol at BACsof 0.5 g/L and above, and for illicit drugs. The results also show thatstandardization of case–control designs will probably not lead tocomparable odds ratios and that further action is needed.

The results of this study reflect the importance for system-atic review studies or meta-analyses of epidemiological studiesthat estimate the risk of driving under the influence of psychoac-tive substances to include assessments of potential errors, suchas described by the Cochrane collaboration (Higgins and Altman,2008). These assessments are essential for understanding therelationship between observed and actual risk estimations. Fur-thermore, we advise that future case–control studies in the field ofdriving under the influence include a pilot study with an assess-ment on potential bias. This would allow identification of thepresence of potential limitations in the study design that couldresult in random and systematic errors. The list of potential indica-tors that was used in this study could be used as guidance, as long asit is kept in mind that this list is tailored to the DRUID case–controlstudies. Therefore, including additional variables in this list mightbe necessary. Finally, an a priori calculation of sample size couldprovide valuable information in how to maximize the precision ofthe study given certain limitations regarding resources and time.

4.1. Strengths and limitations

The main strength of this study within the DRUID-project is thatit provides insight in the source of errors in case–control studies toinvestigate the injury risk of driving under the influence of psy-

choactive substances. The results may also be valuable for futurereviews of similar case–control studies. Another strength of thisstudy is that it allowed the comparison of studies that includedthe same set of psychoactive substances at the same cut-off levels.

1 sis an

Tr

owmnmbmaroa

wFttsstMlteisnsansHol

tsp

4

gntestspdtse

D

Ub65

52 S. Houwing et al. / Accident Analy

his made it possible to examine the differences between the oddsatios.

A limitation of this study was the lack of relevant informationn the non-response group of the case populations. Therefore, itas not possible to detect non-response bias, although such a biasay well have been present. Another limitation is the qualitative

ature of the method of assessment. The results provide at the veryost information on the direction of the bias, but not on the exact

ias between observed and real odds ratios. Furthermore, no infor-ation was available for the storage time of the samples before

nalysis. Degradation of analytes could well have influenced theesults of the surveys (Langel et al., 2008). Reporting informationn the storage time of samples may be of added value as well whenssessing studies for potential bias.

Finally, the numbers of samples from seriously injured driversere very small in some of the national case–control studies (e.g.

inland). A large number of cases is important for the precision ofhe study. However, in some countries the accident rates are lowerhan in other countries, which could make it difficult to collect aufficient number of cases. In order to increase the number of caseamples it may be recommendable to enlarge the study base orhe period of sample inclusion, or to include injured drivers with a

AIS score of 1 (light injury) in future as well in case of potentialow cell counts, under the assumption that there is no clear rela-ionship between injury severity and the presence of drugs (Sminkt al., 2005). To what extent the number of samples needs to bencreased depends on the purpose of the study. A very large sampleize of thousands of cases and controls (Kelsey et al., 1996) would beeeded for calculating significant elevated risks for low odds ratiosuch as the one for cannabis. These numbers were not feasible forny of the national case–control studies, although the combinedumber of cases and controls for all six studies would have beenufficient if the studies had been conducted as originally planned.owever, in practice most studies did not meet their target numberf samples because of practical, financial, medical ethical and legalimitations.

If the main aim of a study is to qualitative rather than quantita-ive, e.g. if the focus would be on determining which psychoactiveubstances have the highest risks, a smaller sample size wouldrobably be sufficient.

.2. Conclusions

The results provide clear indications that in spite of uniformuidelines differences between the odds ratios in the variousational DRUID case–control studies may indeed be at least par-ially explained by random and systematic errors. In general, mostrrors in this study can be attributed to selection bias and to smallample sizes and cell counts. Therefore, the authors recommendhat in epidemiological studies assessing the risk of psychoactiveubstances in traffic special attention should be given to avoid theseotential sources of bias. The list of potential indicators that waseveloped in this study may be useful both as guidance for sys-ematic reviews and meta-analyses and for future epidemiologicaltudies in the field of driving under the influence to minimize theffect of potential sources of errors.

isclaimer

This article has been produced under the project Driving

nder Influence of Drugs, Alcohol and Medicines (DRUID) financedy the European Community within the framework of the EUth Framework Program (contract no. TREN-05-FP6TR-S07.61320-18404-DRUID). This article reflects only the authors’ view. The

d Prevention 52 (2013) 144– 153

European Commission is not liable for any use that may be madeof the information contained therein.

Acknowledgements

We would like to thank all persons that were involved in theroadside surveys and hospital studies in the six countries. Further-more, our thanks go to the staff of the toxicological laboratories whohave analyzed the data. Special thanks go out for Tom Blencowe forediting the final text of the article. Finally we would like to thankthe statisticians from the Danish Technical University for their rolein the calculation of the odds ratios and Dr. Volker Hargutt for hisvaluable input in the discussions on the accuracy of the data.

References

Assum, T., Frison, G., Hels, T., Houwing, S., Mathijssen, R., 2007. Uniformdesign and protocols for carrying out case–control studies. DRUID DRivingUnder the Influence of Drugs, Alcohol and Medicines, D2.1.2 Available fromhttp://www.druid-project.eu/

Blows, S., Ivers, R.Q., Connor, J., Ameratunga, S., Woodward, M., Norton, R., 2004.Marijuana use and car crash injury. Addiction 100, 605–611.

Borkenstein, R.F., Crowther, R.F., Shumate, W.B., Ziel, W.B., Zylman, R., 1974. The roleof the drinking driver in traffic accidents (the grand rapids study). Blutalkohol11 (Suppl. 1), 2nd ed.

Brault, M., Dussault, C., Bouchard, J., Lemire, A.M., 2004. The contribution of alcoholand other drugs among fatally injured drivers in Quebec: final results. In: 17thICADTS Conference, Glasgow.

Cohen, J., 1988. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.Lawrence Earlbaum Associates, Hillsdale, NJ.

Compton, R.P., Blomberg, R.D., Moskowitz, H., Burns, M., Peck, R.C., Fiorentino, D.,2002. Crash risk of alcohol impaired driving. In: 16th ICADTS Conference, Mon-treal.

Connor, J., Norton, R., Ameratunga, S., Jackson, R., 2004. The contribution of alcoholto serious car crash injuries. Epidemiology 15 (3), 337–344.

DRUID, 2012. DRUID DRiving Under the Influence of Drugs, Alcohol and Medicines.Final Report. Available from http://www.druid-project.eu/

Garthe, E., States, J.D., Mango, N.K., 1999. Abbreviated injury scale unification: thecase for a unified injury system for global use. Journal of Trauma 47 (2), 309–323.

Hargutt, V., Krüger, H.-P., Knoche, A., 2011. Driving under the influence of alcohol,illicit drugs and medicines: risk estimations from different methodologi-cal approaches. DRUID DRiving Under the Influence of Drugs, Alcohol andMedicines, D1.3.1. Available from http://www.druid-project.eu/

Haworth, N., Vulcan, P., Bowland, L., Pronk, N., 1997. Estimation of Risk Factorsfor Fatal Single Vehicle Crashes. Monash University Accident Research Centre,Victoria.

Hels, T., Bernhoft, I.M., Lyckegaard, A., Houwing, S., Hagenzieker, M., Legrand, S.-A., et al., 2011. Risk of injury by driving with alcohol and other drugs. DRUIDDRiving Under the Influence of Drugs, Alcohol and Medicines, D2.3.5. Availablefrom http://www.druid-project.eu/

Higgins, J.P.T., Altman, D.G., 2008. Assessing Risk of Bias. John Wiley & Sons Ltd.,http://dx.doi.org/10.1002/9780470712184.ch8 (Chapter 8).

Houwing, S., Hagenzieker, M., Mathijssen, R., Bernhoft, I.M., Hels, T., Janstrup, K.,et al., 2011. Prevalence of alcohol and other psychoactive substances in driversin general traffic. Part 1: General results and Part 2: Country reports. DRUIDDRiving Under the Influence of Drugs, Alcohol and Medicines, D2.2.3. Availablefrom http://www.druid-project.eu/

Houwing, S., Mathijssen, R., Brookhuis, K.A., 2009. Case–control studies. In: Verster,J.C., Pandi-Perumal, S.R., Ramaekers, J.G., De Gier, J.J. (Eds.), Drugs, Driving andTraffic Safety. Birkhäuser Verlag, Basel, pp. 107–120.

Houwing, S., Smink, B.E., Legrand, S.-A., Mathijssen, M.P.M., Verstraete, A.G.,Brookhuis, K.A., 2012. Repeatability of THC measurements for oral fluid collec-tion methods. Forensic Science International 223, 266–272.

Isalberti, C., Van der Linden, T., Legrand, S.-A., Verstraete, A., Bernhoft, I.M., Hels, T.,et al., 2011. Prevalence of alcohol and other psychoactive substances in injuredand killed drivers. DRUID DRiving Under the Influence of Drugs, Alcohol andMedicines, D2.2.5. Available from http://www.druid-project.eu/

Kelsey, J.L., Whittemore, A.S., Evans, A.S., Thompson, W.D., 1996. Methods in Obser-vational Epidemiology, 2nd ed. Oxford University Press, New York.

Kleinbaum, D.G., Kupper, L.L., Morgenstern, H., 1982. Epidemiologic Research. Prin-ciples and Quantitative Methods. Belmont.

Krüger, H.-P., Hargutt, V., Schnabel, E., Brokhuis, K., 2008. Theoretical frame-work for substance effects on safe driving. DRUID deliverable 1.1.1,www.druid-project.eu

Krüger, H.-P., Vollrath, M., 2004. The alcohol-related accident risk in Germany: pro-cedure, methods and results. Accident Analysis & Prevention 36, 125–133.

Langel, K., Engblom, C., Pehrsson, A., Gunnar, T., Ariniemi, K., Lillsunde, P., 2008.Drug testing in oral fluid—evaluation of sample collection devices. Journal ofAnalytical Toxicology 32 (6), 397–401.

Mathijssen, M.P.M., Houwing, S., 2005. The prevalence and relative risk of drinkand drug driving in the Netherlands: a case–control study in the Tilburg police

sis an

M

M

M

M

O

O

P

http://www.druid-project.eu/

S. Houwing et al. / Accident Analy

district. SWOV Stichting Wetenschappelijk Onderzoek Verkeersveiligheid, R-2005-9. Leidschendam.

athijssen, M.P.M., Twisk, D.A.M., 2001. Opname en afbraak van alcohol in hetmenselijk lichaam. SWOV Stichting Wetenschappelijk Onderzoek Verkeersvei-ligheid, R-2001-19. Leidschendam.

elethil, S.K., 2011. Breath Tests for Blood Alcohol Determination: Partition Ratio.http://www.forensic-evidence.com/site/Biol Evid/Breath Tests.html

ovig, K.L.L., Mathijssen, M.P.M., Nagel, P.H.A., van Egmond, T., de Gier, J.J., Leufkens,H.G.M., et al., 2004. Psychoactive substance use and the risk of motor vehicleaccidents. Accident Analysis & Prevention 36 (4), 631–636.

ura, P., Kintz, P., Ludes, B., Gaulier, J.M., Marquet, P., 2003. Comparison of theprevalence alcohol, cannabis and other drugs between 900 injured drivers and900 control subjects: results of a French collaborative study. Forensic ScienceInternational 133, 79–85.

’Neal, C.L., Crouch, D.J., Rollins, D.E., Fatah, A.A., 2000. The effects of collection meth-ods on oral fluid codeine concentrations. Journal of Analytical Toxicology 24 (7),536–542.

ECD, 2010. Drugs and Driving: Detection and Deterrence. OECD Organisation forEconomic Co-operation and Development.

il, K., Esposito, F.M., Verstraete, A., 2010. External quality assessment of multi-analyte chromatographic methods in oral fluid. Clinica Chimica Acta 411(15–16), 1041–1045.

d Prevention 52 (2013) 144– 153 153

Ravera, S., van Rein, N., de Gier, J.J.de, Jong-van den Berg, L.T.W., 2011. Road trafficaccidents and psychotropic medication use in the Netherlands: a case–controlstudy. British Journal of Clinical Pharmacology 72 (3), 505–513.

Rothman, K.J., 1986. Modern Epidemiology. Little, Brown & Co., Boston.Rothman, K.J., Greenland, S., 1998. Modern Epidemiology, 2nd ed. Lippincott-Raven,

Philadelphia.Schmidt, C.A., Kohlmann, T., 2008. When to use the odds ratio or the relative risk?

International Journal of public health 53 (3), 165–167.Schulz, K.F., Grimes, D.A., 2002. Case–control studies: research in reverse. The Lancet

359 (9304), 431–434.Smink, B.E., Ruiter, B., Lusthof, K.J., De Gier, J.J., Uges, D.R.A., Egberts, A.C.G., 2005.

Drug use and the severity of a traffic accident. Accident Analysis & Prevention37 (3), 427–433.

Verstraete, A., Knoche, A., Jantos, R., Skopp, G., Gjerde, H., Vindenes, V., et al., 2011.Per se limits – methods of defining cut-off values for zero tolerance. DRUIDDRiving Under the Influence of Drugs, Alcohol and Medicines. Available from

Wacholder, S., McLaughlin, J.K., Silverman, D.T., Mandel, J.S., 1992. Selection of con-trols in case–control studies. American Journal of Epidemiology 135, 1019–1050.

Walsh, J.M., Verstraete, A.G., Huestis, M.A., Mørland, J., 2008. Guidelines for researchon drugged driving. Addiction 103 (8), 1258–1268.