Best practices in vehicle stop data collection and analysis

Best practices in vehicle stop datacollection and analysis

Rob TillyerUniversity of Texas at San Antonio, San Antonio, Texas, USA

Robin S. EngelUniversity of Cincinnati, Cincinnati, Ohio, USA, and

Jennifer Calnon CherkauskasUniversity of Cincinnati, Sierra Vista, Arizona, USA

Abstract

Purpose – Within the last 15 years, law enforcement agencies have increased their collection of dataon vehicle stops. A variety of resource guides, research reports, and peer-reviewed articles haveoutlined the methods used to collect these data and conduct analyses. This literature is spread acrossnumerous publications and can be cumbersome to summarize for practical use by practitioners andacademics. This article seeks to fill this gap by detailing the current best practices in vehicle stop datacollection and analysis in state police agencies.

Design/methodology/approach – The article summarizes the data collection techniques used toassist in identifying racial/ethnic disparities in vehicle stops. Specifically, questions concerning why,when, how, and what data should be collected are addressed. The most common data analysistechniques for vehicle stops are offered, including an evaluation of common benchmarking techniquesand their ability to measure at-risk drivers. Vehicle stop outcome analyses are also discussed,including multivariate analyses and the outcome test. Within this summary, strengths andweaknesses of these techniques are explored.

Findings – In summarizing these approaches, a body of best practices in vehicle stop data collectionand analysis is developed.

Originality/value – Racial profiling continues to be a contentious issue for law enforcement and thecommunity. A considerable body of research has developed to assess the prevalence of racial profiling.This article offers social scientists and practitioners a comprehensive, succinct, peer-reviewedsummary of the best practices in vehicle stop data collection and analysis.

Keywords Police, Decision making, Bias, Discrimination, Traffic enforcement

Paper type General review

1. IntroductionPolice decision making prior to and during vehicle stops has garnered considerableattention in the last 15 years. Developing out of two high profile legal cases[1,2],

The current issue and full text archive of this journal is available at

www.emeraldinsight.com/1363-951X.htm

This research is based on information gathered for the Traffic Stop Data Analysis Study,directed by Dr Robin Engel, and supported by the Arizona Department of Public Safety (ContractNo. L7-009). Points of view in this document are those of the authors and do not necessarilyrepresent the official position or policies of the Arizona Department of Public Safety. Pleasedirect any correspondence regarding this contract to Robin S. Engel, PhD, Division of CriminalJustice, University of Cincinnati, PO Box 210389, Cincinnati, OH 45221 (e-mail:robin.engel@uc.edu). For correspondence regarding this paper, see details at the end of thearticle.

Best practices invehicle stop data

collection

Received 20 January 09Revised 9 July 2009

Accepted 15 July 2009

Policing: An International Journal ofPolice Strategies & Management

Vol. 33 No. 1, 2010pp. 69-92

q Emerald Group Publishing Limited1363-951X

DOI 10.1108/13639511011020601

law enforcement agencies, policymakers, the legal community, academics, and citizensfocused specific attention on vehicle stops to determine if racial/ethnic disparities wereoccurring in police decisions to stop motorists and/or in vehicle stop outcomes (Buergerand Farrell, 2002; General Accounting Office, 2000; Harris, 2002; 2006). To inform thisissue, numerous data collection efforts were initiated by law enforcement agencies (Fridell,2005; Fridell et al., 2001; Hickman, 2005; Institute on Race and Poverty, 2001; Strom et al.,2001). To date, analyses of these data demonstrate a relatively consistent trend ofracial/ethnic disparities in vehicle stops and vehicle stop outcomes (e.g. Alpert Group,2004; Alpert et al., 2006; Engel et al., 2007b; Farrell et al., 2004; Ingram, 2007; Lamberth,2003; Lovrich et al., 2005; Ridgeway et al., 2006). Such patterns of disparity do notnecessarily indicate that law enforcement officers are relying on race/ethnicity as the onlyfactor in decision making; however, it does highlight the importance of examining thesepatterns to better understand the relationship between citizens’ race/ethnicity and officerdecision making within the context of vehicle stops. Moreover, vehicle stops are the mostcommon type of police-citizen encounter (Durose et al., 2007); thus, decision making withinthis context has important implications for police-citizen relations.

As previous commentators have noted, data collection and analysis of vehicle stopdata are frequently characterized by methodological and statistical limitations thatneed to be recognized and, if possible, addressed (Engel and Calnon, 2004; Fridell, 2004,2005; Fridell et al., 2001; Ramirez et al., 2000). Failing to acknowledge such limitationsof current data collection and analysis will lead to conclusions that at best lackempirical rigor, and at worst lead to serious legal and policy ramifications. Whenproperly analyzed and interpreted, vehicle stop data can assist in identifyingracial/ethnic disparities, offer insight for police administrators, and guide policy andtraining modifications to enhance equitable policing practices across racial/ethnicgroups (Engel et al., 2007a; Fridell, 2004). Therefore, a thorough understanding andexploration of the strengths and weaknesses of vehicle stop data collection andanalysis is crucial to the proper interpretation of vehicle stop findings.

These issues have been explored in a variety of resources guides (e.g., Fridell, 2004,2005; Fridell et al., 2001; Ramirez et al., 2000), technical agency reports (e.g. AlpertGroup, 2004; Engel et al., 2007a; Lovrich et al., 2003, 2005), and peer-reviewed academicliterature (e.g. Batton and Kadleck, 2004; Engel, 2008; Engel and Calnon, 2004; Walker,2001). Collectively, these publications address many of the methodological andstatistical issues involved in vehicle stop data collection and analysis. Unfortunately,this body of knowledge is scattered across multiple sources and does not succinctlyidentify current best practices in vehicle stop data collection and analysis.

This article aims to fill this void by providing a comprehensive summary[3] of thecurrent best practices in vehicle stop data collection and analysis within state policeagencies[4]. Initially, the main considerations for effective vehicle stop data collectionare reviewed including why state police agencies collect data, what types of situationsare typically captured in these data, the options available for collecting thisinformation, and what information should be collected regarding these encounters.Thereafter, a review of vehicle stop data analysis is provided including an overview ofvarious analytical techniques and their associated strengths and weaknesses.Collectively, this article provides a comprehensive summary of the most salient issuesin vehicle stop data collection and analysis, highlights promising methodologies for

PIJPSM33,1

vehicle stop data collection and analysis, and offers recommendations for identifyingracial/ethnic disparities in vehicle stops and vehicle stop outcomes.

2. Vehicle stop data collectionData collection offers an agency the opportunity to assess quantitatively andempirically the work behavior of its members in order to better understand vehicle stopdecision-making (Ramirez et al., 2000; Fridell et al., 2001). The scope and content of datacollection is crucial to identify potentially problematic areas or officers who requireincreased scrutiny and perhaps departmental intervention. In this manner, datacollection has several strengths, including as a tool for developing training practicesand policies regarding acceptable officer behavior (Davis et al., 2001; Fridell, 2004;Fridell et al., 2001; Ramirez et al., 2000). Data collection can also serve as a foundationfor opening a dialogue between law enforcement agencies and communities. As aresult, data collection assists in developing a body of research that informs the legalsystem, police, politicians, and citizens regarding patterns or trends of potentialracial/ethnic disparities in police decision-making.

Despite its strengths, data collection efforts also have potential negative side effectsfor the agency. First, law enforcement agencies are often concerned that data collectionmay become a burden upon officers and the agency more generally (Ramirez et al.,2000). Cumbersome or inconvenient requirements for data collection, officerdisengagement due to poor officer morale, budgetary startup and maintenance costsfor data collection and/or unanticipated budgetary constraints all may contribute tosuch concerns. In any of these situations, the integrity of the data collection effort maybecome compromised. Second, if the inappropriate use of race/ethnicity is discovered,the agency may have to discipline its member and/or changes in training may berequired. Such activities may negatively affect officer morale or require additionalfinancial costs for new training curriculum; however, ultimately the outcomes fromchanges in training, policies and procedures may benefit law enforcement agencies.Finally, analyses of data may be used against an agency in selective enforcementlitigation (Smith and Alpert, 2002). All these factors actively contribute to some lawenforcement agencies’ reluctance to collect vehicle stop data.

Overcoming these limitations frequently requires a comprehensive and reasonableplan for data collection. A first and crucial step in the development of a data collectionplan often involves a task force or working committee comprised of officers, citizens,and researchers (Fridell, 2004, Fridell et al., 2001; Ramirez et al., 2000). Other entitiesmay also be involved but should only be included if they have expertise or a vestedinterest in the data collection effort. When developing a plan, this group should attemptto answer the following four questions:

(1) Why are data being collected?

(2) When should data be collected?

(3) How are the data to be collected?

(4) What information should be collected?

The issues surrounding each of these questions are detailed below.

collection

2.1 Why collect data?Identifying and understanding the motivations for data collection can assist indeveloping an effective plan. Data collection efforts to uncover racial/ethnic disparitiesin vehicle stops are generally motivated by a variety of forces, including as arequirement of litigation, due to a legislative mandate, and/or as a result of proactiveaction by the law enforcement agency to address community concerns (Davis, 2001;Davis et al., 2001; Ramirez et al., 2000). This latter factor is particularly salient inindicating a commitment to accountability by the agency and communicates a messageof legitimacy to the community (Farrell et al., 2003, 2004; Fridell, 2004; Fridell et al.,2001; Ramirez et al., 2000).

In recent history, claims of bias against law enforcement agencies or its memberswere sometimes resolved through litigation, which often were a catalyst for datacollection efforts[1,2]. Using these cases as a template, the resolution of racial/ethnicdisparity claims may be dictated by court order or a settlement between the plaintiffand the agency and often require agencies to collect information on all vehicle stops bymandating the method of data collection and the specific data fields to be collected(Smith and Alpert, 2002). These arrangements generally offer little flexibility for theagency. Often, relevant information may not be captured as part of the required datacollection (i.e. reasons for the stop, behavior of the drivers/passengers, an appropriatebenchmark, etc.) (see, for example, Farrell et al., 2003, 2004; Rojek et al., 2004; Withrow,2004).

Legislative mandates also underpin data collection efforts developed within the last15 years. It has become commonplace for legislation to couple the requirement of datacollection with a statement banning the use of race/ethnicity in vehicle stop decisions(Hickman, 2005; National Conference of State Legislatures, 2001; Ramirez et al., 2000;Strom et al., 2001; Strom and Durose, 2000). In 2007, 17 states legislatively requiredtheir state highway police/patrol to collect vehicle stop data. In addition, 11 states hadlegislation prohibiting racial profiling, another eight states had bills underconsideration, and 14 states either did not have explicit legislation (n ¼ 11) or hadmore complex nuanced policies regarding racial profiling data collection (n ¼ 3)[5].The legislative mandates for data collection are displayed graphically in Figure 1.

This type of statewide, enforced data collection presents challenges to lawenforcement agencies. Under a legal mandate, a “one-size fits all” approach to datacollection may be adopted which requires all agencies throughout the state (i.e. bothstate and local) to collect identical information on vehicle stops regardless of whetheror not this information is pertinent to the agency (e.g. Farrell et al., 2004). Theinformation mandated for collection is often so basic that it does not allow for detailedanalysis that could actually be beneficial to law enforcement agencies. Further,legislation often does not require and/or provide recommendations regarding dataanalysis. In such situations, officer morale may suffer or officers may discontinueinitiating vehicle stops (i.e. officer disengagement) (Ramirez et al., 2000).

It is recommended that police agencies collecting vehicle stop data as a result oflitigation or legislation approach data collection as an opportunity to better understandthe vehicle stop practices of its officers. These agencies should consider adopting datacollection fields that are not required under court agreement or statute, but that offermore comprehensive information regarding officer decision-making within the contextof vehicle stops. Moreover, agencies that voluntarily collect additional data on vehicle

PIJPSM33,1

stops may assist in developing a positive rapport with their constituents bydemonstrating a willingness to investigate their own behavior to ensure that they aretreating all citizens fairly.

Finally, some agencies chose to voluntarily collect race/ethnicity information duringvehicle stops, albeit sometimes with stakeholder influence. This process often involvesa partnership with an independent researcher(s) to assist in the data collection andanalysis. Proactively collecting vehicle stop data can offer advantages to agenciesincluding the ability to decide on the specific types of information to be collected, andthe freedom to assess the data based on the analysis technique of their choosing. Arecent review of state police agencies currently collecting vehicle stop data or agenciesthat collected data in the past revealed that 18 of 34 agencies (52.9 percent) voluntarilyinitiated the data collection effort (Engel et al., 2007a).

2.2 When to collect data?Regardless of the impetus for data collection, decisions must be made regarding thetypes of police-citizen encounters that will be included in the official data collectioneffort. In some situations, data are only collected on vehicle stops that result in aspecific outcome such as an arrest or search, but information is not collected on vehiclestops that result in a warning. These decisions may be predetermined when datacollection efforts are initiated due to legal or legislative mandate. If there is flexibility indata collection, however, the agency should strive to balance the competingconsiderations of officer workload versus the necessity of collecting enoughinformation to allow robust analysis (Engel et al., 2007a; Fridell et al., 2001; Ramirezet al., 2000). Based on these factors, data collection can be broadened or narrowed.

Figure 1.Nationwide legislation

collection

If the agency wishes to focus on vehicle stops, it is recommended at a minimum thatdata be collected during all officer-initiated vehicle stops, regardless of the outcome.Collecting information during all officer-initiated vehicle stops focuses the analyses onofficer-decision making. When data collection and analyses include dispatched andcitizen-initiated vehicle stops (e.g. accidents), it is difficult to determine the impact ofofficer decision making on initial stop decisions. If this information is collected,non-officer initiated vehicle stops should be excluded from the analyses that seek toexplain officers’ discretion in initial stop decisions.

2.3 How to collect data?The method of data collection plays a crucial role in determining if the data are collectedaccurately and what type of information is captured. Various methods exist for datacollection (Fridell, 2004), but most frequently it is accomplished through one of fourmethods. The most common approach is to use a paper form that is completed by officersaccording to the protocols of the data collection methodology (Hickman, 2005). Agencypersonnel then enter the data manually into an electronic database. This requires no newsoftware or hardware; however, it does require extra manpower to transfer the data intoan electronic format. It also takes considerable time to process the data, and the data errorrate may be inflated due to data entry errors or an inability to read officers’ handwriting.

Alternatively, paper-based scanned forms are used to collect the information. Thesedata are transferred into an electronic database by a scanner rather than throughhuman data entry. This method is generally more efficient and is relatively easy toimplement and use. This process does require the purchase of scanning equipment andthe purchase or printing of specific data collection forms. Limitations of this methodinclude the introduction of additional paperwork for officers, the potential for anincrease in error rates associated with new paperwork, and the forms may restrict thetype of data that can be collected due to budgetary considerations.

A third option involves an officer directly entering the required information into acomputer or mobile data terminal (MDT) at the conclusion of the encounter. Thismethod is advantageous because there is no delay in accessing and analyzing the data,and it reduces the amount of paperwork as multiple reports can be developed fromsingle data entries. This system also offers a significant reduction in errors becauseofficers are required to fill out the form correctly prior to submission. Limitations ofthis method include the need for a significant amount of start-up money and theincreased burden on support services within the agency due to the use of technology.

Finally, some municipal agencies simply collect information by officers reporting itorally to dispatchers, and dispatchers recording it in paper or electronic format. While thisis the easiest form of data collection for officers, it may become burdensome for dispatchersin larger jurisdictions. Further, in jurisdictions with higher volumes of radio traffic, thisapproach is likely not a viable option and is not routinely used by state police agencies.

In 2007, 21 of the 34 state police agencies collecting information on vehicle stopsused a paper-based form. Of those 21 agencies, 15 agencies use manual data entrypaper forms, while six of the agencies have adopted a scanner based system. Five ofthe 34 state police agencies collecting vehicle stop data directly entered informationinto the electronic database through the vehicle MDT. Four of the 34 agencies use acombination of paper-based and electronic data collection forms, while one agency usesits computer-aided dispatch (CAD) system for data collection. Finally, two of the six

PIJPSM33,1

state agencies that used a scanner-based system in 2007 (i.e. Arizona Department ofPublic Safety and Pennsylvania State Police) changed to an electronic system in 2008.Information regarding the method of data collection was not available for theremaining three state police agencies.

From a data reliability perspective, instituting an electronic data capture system is thebest option. This system records the information entered by the officer on the vehicle’sMDT and then transmits that information directly to a central data repository for storageand future analysis. This approach offers many advantages, including low error rates,the ability to immediately analyze the data, and no additional paperwork for officers. Ifan electronic capture system is not possible, other methods need to be coupled withstrong supervisory oversight and unwavering support from the agency’s command staff.

Regardless of the data collection method adopted, all data collection efforts shouldinclude a pilot test and continuous, regular data audits. A pilot test should beconducted to identify any difficulties with the implementation of the system andaddress any further officer training needs. This trial run offers the ability to correctsystematic errors in data collection, thus reducing long-term problems. An agencymust also know and maximize the quality of its data in order to be confident in theresults of analysis.

Many recommendations offered by other experts in regard to data analysis areapplicable to data auditing. For example, Fridell (2004) notes the value of independentanalysts during the analysis of vehicle stop data because they offer methodological andstatistical expertise, and this point has resonance in regard to data auditing as well.Moreover, independent analysts may also infuse a degree of credibility to the researchdue to their independence and objectivity during the data analysis stage (Fridell, 2004);similarly, data audits offer an assessment of the data by individuals who did not collectthe data. Most guides on vehicle stop research advocate a partnership between theagency and researcher that is initiated early in the development phase of the researchand that continues throughout the analysis and reporting stages (Davis, 2001; Daviset al., 2001; Engel et al., 2007a; Fridell, 2005, 2004; Fridell et al., 2001; McMahon et al.,2002, Ramirez et al., 2000). Again, the utility of partnerships between researchers andagencies would be manifest in the data auditing process. Finally, officers are morelikely to be diligent in data collection when they are monitored (Fridell, 2004).

2.4 What data to collect?One of the most important considerations for data collection is the determination ofwhat data to collect. State police agencies vary considerably on the type of informationcollected; however, to ensure that data collection is comprehensive, specificcharacteristics of the vehicle stop need to be recorded. Data collection isstrengthened by the measurement of multiple factors so that the analyses canaddress various hypotheses regarding the vehicle stop and vehicle stop outcome.

Vehicle stops present situations in which a multitude of variables operatesimultaneously to potentially influence officer decision making and vehicle stopoutcomes. These variables can be generally grouped into stop, driver, vehicle, andofficer characteristics (i.e. independent variables), while the outcome of the vehicle stop(i.e. a warning, a citation, an arrest, a search, etc.) is frequently the dependent variable.Collecting information on these factors allows for analysis of officer decision making inpolice-citizen encounters.

collection

Stop characteristics provide contextual information regarding the stop and shouldinclude the time and date of the stop, the location where the stop occurred[6], and thereason for the stop[7]. Data fields should also collect information on the duration of thestop and if the citizen was warned, cited, or arrested. More intricate data collectionefforts also specify multiple violations and their related outcomes. Most recently, ahandful of state and local agencies are considering the documentation of motorists’behaviors observed prior to the stop that are viewed as suspicious and suggestive ofpossible illegal behavior. These “pre-stop” indicators are those suspicious behaviorsthat are often considered for pre-textual stops, but are not the legal reason for thevehicle stop. At least one state police agency (Arizona Department of Public Safety) isnow collecting the presence of pre-stop indicators of suspicion. It is important toconsider these variables for inclusion in data collection efforts. If there are racial/ethnicdifferences in the presence of pre-stop indicators, this may explain some of theracial/ethnic disparities evident in the search rates of most police agencies across thecountry. Based on focus group interviews with officers specializing in criminalinterdiction work, it is recommended that the following information regardingsuspicious behavior be collected:

. vehicle type/condition/modifications;

. driver’s body language;

. occupants’ behaviors;

. driving behavior; and

. other, with a default of “none” and the option for officers to mark all categoriesthat apply.

Information on whether or not a search was conducted, the reason for the search, and ifany property or evidence was seized are also important. In particular, search andseizure information is important for the agency so they can assess the quantity andquality of searches conducted and their productivity in terms of contraband (Fridellet al., 2001; Ramirez et al., 2000). Some state agencies have attempted to collect searchand seizure information that is specific to each individual search target (e.g. driver,vehicle, passenger, or pedestrian). This type of search-related data collection allows forthe most accurate analysis of searches and search success rates based on differentlevels of officer discretion.

Driver characteristics should include the race/ethnicity, gender, and age of thedriver. The race/ethnicity of the driver should be recorded based on the officer’sperception rather than directly asking the citizen because concerns of racial bias stemfrom the officer’s perception of race/ethnicity (Davis, 2001; Fridell et al., 2001; Ramirezet al., 2000). Drivers’ gender and age is often gathered from their driver’s licenses toensure its accuracy. Officers’ perceptions of drivers’ gender and age have not beenraised in this body of literature as a concern. Drivers’ license information can alsoindicate if the individual is a local resident (Fridell et al., 2001; McMahon et al., 2002).One important data field that is not collected on a regular basis is the demeanor of thedriver. Past research has demonstrated the impact of this variable on officer decisionmaking (see, for example, Lundman, 1996, 1994; Worden and Shepard, 1996), but it iscurrently included in only a few vehicle stop studies (e.g. Engel et al., 2010).

PIJPSM33,1

Vehicle characteristics should include the license plate number and/or the state ofvehicle registration. In some cases, the type, condition, and/or age of the vehicle are alsorecorded (e.g. Engel et al., 2006). Although they are not frequently collected, these datafields are important to consider for inclusion because some of the disparity in vehicle stopoutcomes may be due to socio-economic status rather than race/ethnicity. Characteristicslike vehicle condition and age are the best available proxy indicators of wealth andsocioeconomic status that are likely to be included on a vehicle stop data collection form.

An officer identifier should be recorded on the vehicle stop form to assess whetherspecific officer characteristics are associated with disparity in decision-making (Engelet al., 2007a). For example, an officer’s age, gender, race/ethnicity, assignment,education, and experience may be correlated with decision making. Collecting anofficer identifier on all vehicle stops is controversial and may raise concerns. Officersmay fear that the information will be used against them in agency discipline or infuture litigation. Agencies may be reluctant to collect this information if the unionopposes such a data field, if organizational morale suffers, or if there is a possibility ofofficer disengagement. These limitations need to be weighed against the benefits ofunderstanding the officer’s role in decision making.

3. Vehicle stop data analysesData analysis is the second phase of a vehicle stop study. The primary goal of dataanalysis is to identify racial/ethnic patterns of disparity in the agency’s vehiclestopping practices. To accomplish this, vehicle stop analyses are separated into twocomponents:

(1) the initial decision to stop a vehicle; and

(2) post-stop outcomes (Engel et al., 2004; Novak, 2004; Ramirez et al., 2000; Rojeket al., 2004; Smith and Alpert, 2002).

Analyses of the initial vehicle stop decision require the use of benchmarks, whichintroduce several limitations. Analyses of post stop outcomes, on the other hand, offermore rigorous analysis options such as multivariate models, the outcome test, andpropensity scores.

3.1 Vehicle stop analysesTwo sets of information are necessary to explore whether disparities exist in the initialvehicle stop decision. First, the actual rate of stops by drivers’ race/ethnicity is requiredand is generally provided by the agency’s vehicle stop database. Second, a data sourceis needed that measures the expected rate of stops by race/ethnicity (i.e. a benchmark orbaseline) assuming no police bias exists (Engel and Calnon, 2004; Engel et al., 2007a;Fridell, 2004; Rojek et al., 2004; Smith and Alpert, 2002). A common method ofassessing these data is to develop a disproportionality index or ratio. For eachracial/ethnic group of interest, the vehicle stop data are placed in the numerator and thebenchmark data are entered in the denominator. If the resulting index is above orbelow the value of one, a disparity exists between the two groups.

The difficulty associated with this type of analysis is in attempting to identify abenchmark that accurately reflects the true rate of drivers using the roadways andviolating vehicle laws. As social scientists have refined benchmarking techniques, theyhave reached a general consensus that not all drivers are equally likely to be stopped.

collection

Therefore, a benchmark must reflect the drivers’ risk of being stopped, assuming nobias. Drivers’ risk of being stopped for vehicle infractions or other legal reasons may beimpacted by both legal and extra legal factors. While an exhaustive itemized list ofthese factors would be nearly impossible to create given all of the possiblecircumstances present, a set of dimensions can be conceptualized to assist in theclassification and conceptualization of motorists’ risk of apprehension. Further, thequality of benchmarks can be best evaluated across these dimensions that representthe likelihood of drivers being stopped. Relevant categories of motorists’ risk forvehicles stops include:

. where they drive;

. when they drive;

. how often they drive;

. what they drive;

. how they drive; and

. who they are (Engel et al., 2004, 2005, 2007d; Fridell, 2004)[8].

A variety of benchmarks have been used as proxies for the expected drivingpopulation at risk of being stopped (Engel et al., 2004; Farrell et al., 2003; Fridell, 2004;Fridell et al., 2001). Benchmarks include the use of residential census population data,“adjusted” census population data, drivers’ license data, not-at-fault vehicle accidents,blind enforcement mechanisms, observations of roadway usage and law-violatingbehavior, and internal comparisons (Engel and Calnon, 2004; Fridell, 2004; Fridell et al.,2001; Smith and Alpert, 2002; Walker, 2001).

The specific strengths and weaknesses of these benchmarks have been detailed inseveral previous discussions and are only briefly summarized here (Engel and Calnon,2004; Fridell, 2004; Fridell et al., 2001; Rojek et al., 2004; Tillyer et al., 2008; Walker,2001). Instead, we evaluate the best practices in benchmarking techniques based ontheir ability to measure the six known risk factors for being stopped. Specifically,Table I provides an assessment of each benchmark’s ability to measure:

. driving location;

. time of travel;

. driving frequency;

. vehicle type and condition;

. driving behavior; and

. demographic characteristics.

Table I also includes a brief overview of each measure’s strength and weaknessesbased on previous summaries.

Census data reflect the racial/ethnic composition of residents recorded in the censusas a proxy for the driving population at-risk of being stopped. In some analyses, thecensus data are adjusted to only consider driving aged citizens as the benchmark(Davis, 2001; Ramirez et al., 2000). Additionally, some researchers have developed aweighted spatial model of driving patterns based on the location of business andshopping locations available in census data (Farrell et al., 2004; Novak, 2004; Rojek

PIJPSM33,1

s’ch

s’li

(continued

Table I.Comparison of

benchmarks by riskfactors

collection

s’ch

Table I.

PIJPSM33,1

et al., 2004). The limitations of census-based benchmarks have been frequentlydiscussed (Davis, 2001; Engel and Calnon, 2004; Fridell, 2004, 2005; Fridell et al., 2001;Ramirez et al., 2000), but are obvious based on the assessments provided in Table I.Specifically, census data and adjusted census data only measure one of the six riskfactors (i.e. demographic characteristics) and are premised on the faulty assumptionthat citizens exclusively drive where they live and live where they drive (Engel andCalnon, 2004; Engel et al., 2007a; Walker, 2001). Further, it could be argued thatadjusted census benchmarking measures the additional risk of driving frequency byusing census information that includes only households with vehicle ownership. Thesemeasures, however, are so rudimentary that although they illustrate an attempt tomeasure this risk factor, in reality, the risk remains untapped. While noting that thegoal of adjusted census benchmark is to better identify and measure driving frequency,the techniques and data available simply have not met this threshold for face-validity.

A more exact measure of actual drivers is offered by the use of drivers’ license datadrawn from the local Department of Motor Vehicles (DMV) (Fridell, 2004). Using thisbenchmark, the race/ethnicity of drivers is identified from the drivers’ licenses issuedin the geographic area and compiled to create a benchmark for analysis. Limitations ofthis approach include no measure of driving frequency and no measure of drivingbehavior. Similar to census-based measures, the use of drivers’ license data also onlycaptures driver demographics without the benefit of measuring any of the other at-riskfactors. Again, while the goal is to better measure risk based on frequency of driving,data indicating the population with drivers’ licenses does not adequately measureindividuals’ driving quantity.

Not-at-fault accident data[9] have also been introduced as a benchmark because thenot-at-fault driver’s presence is assumed to be random, thus providing a true measure ofthe driving population (Alpert et al., 2004). The comparison occurs between therace/ethnicity of drivers stopped by police (the numerator) and the race/ethnicity ofdrivers involved in crashes (the denominator) (Alpert et al., 2004; Fridell, 2004; Fridellet al., 2001). One limitation of this benchmark is the limited number of crashes that occurat one location, which may raise reliability and generalizability concerns. Second, mostcrash reports do not collect information on either driver’s race/ethnicity. Despite theselimitations, not-at-fault collision data measure five of the six risk factors, includingdriving location, time of travel, vehicle type and conditions, driving behavior, and drivercharacteristics (see Table I). Benchmark measures of not-at-fault accidents are designed,in part, to tap into risks of apprehension based on driving frequency, based on thepresumption that drivers who travel more often are more likely to be involved innot-at-fault accidents. Nevertheless, as with adjustment census and licensed driversbenchmarks, not-at-fault accidents provide a somewhat crude and likely less reliablemethod of measuring driving quantity. Further, it is possible that the racial/ethniccomposition of not-at-fault drivers does not reflect the racial/ethnic composition of alldrivers. Further research is needed to test these underlying assumptions.

Blind enforcement mechanisms, such as stationary speeding cameras,RADAR/LASER enforcement by aircraft, and nighttime stops, have also been utilizedas proxies for the at-risk driving population (Fridell, 2004). These methods collectbenchmark data with no human knowledge of the drivers’ race/ethnicity. For example,stationary speeding cameras automatically capture driving violations by photographand those photographs are used to record the driver’s race/ethnicity (Lange et al., 2005).

collection

In other situations, information regarding a speeding vehicle is traced back to thelicensed owner to determine the driver’s presumed race/ethnicity[10]. RADAR/LASERenforcement by aircraft identifies law violations prior to the stop and then the driver’srace/ethnicity is recorded once the vehicle stop is initiated (e.g. Lovrich et al., 2003).Finally, nighttime vehicle stops offer a baseline of drivers’ race/ethnicity that is used tocompare against daytime vehicle stops. In more sophisticated analyses, daylight savingstime stops are compared pre/post based on seasonal time changes (described as the“veil-of-darkness” method) (Grogger and Ridgeway, 2006; Riley et al., 2005).

Blind enforcement benchmarks have been used infrequently to date and vary intheir ability to provide a measurement of driving quantity, quality, location, behavior,vehicle type and condition, and driver characteristics. Despite offering the potential tomeasure all at-risk factors, stationary cameras are generally limited by measuring onlyone type of violation (e.g. speeding), their placement in select locations, and dependingon the quality and clarity of the photograph, the ability to ascertain the driver’srace/ethnicity. All types of blind enforcement techniques, including RADAR/LASERand nighttime vehicle stops, do not measure driving quantity.

Roadway observations are another common type of benchmark that involvesplacing observers on the roadway to record the race/ethnicity of the driving population(Fridell, 2004; Fridell et al., 2001; Lamberth, 1994, 1996). Once collected, theseobservations are used as the benchmark against which stop data will be compared.Roadway observations may focus (and collect data) only on drivers who demonstrate adriving violation for which they could legally be stopped (i.e. speeding or illegal lanechanging) (Engel et al., 2004; Smith et al., 2003), or roadway observations may collectinformation on all passing vehicle regardless of vehicle violations[11]. Variations ofthis method have been used, including collecting data through stationary observations(i.e. median or side of road, tollbooths) (Engel et al., 2004, 2006; Lange et al., 2005) androlling observations in which the observers are in a moving vehicle (Lamberth, 1994,1996). Observation methodologies have been used frequently in past research, with thequality of the method varying dramatically across studies (see Engel et al., 2005, 2006;Lamberth, 1994, 1996; Lange et al., 2005; Meehan and Ponder, 2002a, b; Solop, 2004a, b).

As shown in Table I, observations have the potential to measure all six risk factorsif operationalized properly. Indeed, observation data has been accepted in court as areliable measure of the driving population[12-17]. Even when scientific rigor is applied,however, limitations exist for this technique. For example, conducting enoughobservations to generalize to a larger area often requires extensive financial resourcesand time. The collection of drivers’ race/ethnicity may be based on only one form of lawviolating behavior (e.g. speeding) and may ignore other law-violating vehicles.Observations can only be conducted during daylight hours as identifying the driver’srace/ethnicity is severely limited during nighttime hours. Observations are also limitedby the skill level of observers and their degree of agreement on the race/ethnicity of thedriver. Distinguishing between particular racial/ethnic groups can be challenging to aninexperienced observer (i.e. Hispanic drivers versus White drivers, or Hispanic driversversus Black drivers). Thus, while observations have considerable potential as abenchmark, they have several characteristics, which may limit their accuracy.

Finally, internal benchmarking offers a unique approach to measuring the at-riskdriving population (Walker, 2001). In this method, the vehicle stopping behavior ofofficers working similar assignments, shifts, and jurisdictions are used as baseline, and

PIJPSM33,1

each officer’s activity is then compared against that benchmark to identify disparatebehavior (Engel et al., 2007a; Fridell, 2004; 2005). The crucial component to this method ismatching officers by shift and jurisdiction. If this is accomplished, all officers workingthat time and space are presumed to be exposed to similar driving populations. That is,the six factors associated with drivers’ risk of being stopped are presumed to beequivalent. The primary weakness of internal benchmarking is the inability to identifybias if all members of the group are acting in a similar fashion (Walker, 2001).Furthermore, despite matching on assignment, shift, and jurisdiction, it is possible thatofficers are exposed to different driving populations. Finally, for most large and diversestate police agencies that have a small concentration of officers working in any particulararea and assignment, internal benchmarking may simply not be feasible.

The most significant limitation to all of these benchmarks is the inability to measureadequately all of the risk factors associated with the likelihood of being stopped(Fridell, 2004; Fridell et al., 2001; Engel et al., 2004; Tillyer et al., 2008). Moreover,results of vehicle stop analyses can vary dramatically depending on the benchmarksused (Engel and Calnon, 2004). While the social science community has reached ageneral consensus about the least appropriate benchmark measures (i.e. census-baseddata), no such agreement exists regarding benchmarking best practices. Regardless ofthe baseline selected, no comparison data can provide an exact measurement of theat-risk driving population. Using multiple benchmarks may alleviate someshortcomings; however, interpretations and policy implications of data analysesusing benchmarks must be conducted with caution.

Alternative methods of analyzing vehicle stop data, such as citizen surveys, mayadd some clarity to understanding bias in vehicle stops in an effort to triangulate someof these measures (for examples, see Durose et al., 2007; Engel, 2005; Engel and Calnon,2004; Lundman and Kaufman, 2003; Schmitt et al., 2002; Smith and Durose, 2006). Inaddition, vehicle stop data could also be examined across time to identify if stoppingpatterns are consistent (for an example, see Engel et al., 2007d). Both of these methodspresent their own practical and methodological difficulties, but may partially fill thevoid based on the limitations of benchmarking analyses. Currently, vehicle stop studiesthat examine racial/ethnic disparities in initial stopping decisions are severely limited,and definitive conclusions often cannot be made regarding racial/ethnic disparities inthe decision to initiate a vehicle stop.

3.2 Post-stop outcome analysesPost-stop outcome analyses examine the existence of racial and ethnic disparities afterthe vehicle stop has been initiated. Most often, analyses of these post-stop outcomesfocus on warnings, citations, arrests, searches and/or seizures of contraband (Fridell,2004, 2005), but may also include the length of a stop, whether a person was asked toexit the vehicle, and whether a canine was called. A major advantage of examiningpost-stop outcomes is that, unlike vehicle stops where the comparison population isunknown and can only be estimated, the comparison population for post-stop outcomesis known (Engel et al., 2007a; Tillyer et al., 2008). Post-stop outcomes offer the ability toassess the impact of driver characteristics (e.g. race/ethnicity of the driver, age of thedriver), legal considerations (e.g. the reason for the stop, seriousness of the offense,discovery of contraband), officer characteristics (e.g. length of service, education level,etc.), stop characteristics (e.g. time of day, location of the stop), and the characteristics

collection

of the geographic location where the stop occurred (e.g. crime rate of the neighborhood,racial composition of the neighborhood.). Various techniques, such as multivariateanalyses or the outcome test, exist to identify racial/ethnic disparities in vehicle stopoutcomes.

A multivariate statistical model can be used to assess multiple different vehicle stopoutcomes while controlling for other potentially important factors beyond therace/ethnicity of the citizen (Fridell, 2004, 2005; McMahon et al., 2002; Ramirez et al.,2000). Thus, greater confidence can be placed in multivariate results regardingracial/ethnic disparities compared to the results gained from bivariate analyses(Fridell, 2004, 2005; McMahon et al., 2002; Ramirez et al., 2000). The results ofmultivariate analysis are relatively easy to interpret (Engel et al., 2007a), and pastresearch has consistently used multivariate analysis as a method to assess vehicle stopoutcomes (e.g. Alpert et al., 2006; Engel et al., 2004, 2005, 2006, Alpert et al., 2004;Gumbhir, 2004; Ingram, 2007; Lovrich et al., 2003, 2005; Schafer et al., 2006; Withrow,2004). Multivariate analysis does possess limitations, however, which must beconsidered. One of the primary shortcomings of multivariate analysis is specificationerror, in which other important variables are not measured and included in the analysis(Hanushek and Jackson, 1977). Specification error may lead to inaccurate results if keyfactors are not included. In the case of vehicle stop outcomes, driver demeanor may bea key factor that is often not collected or analyzed (Engel et al., 2010).

An important consideration for multivariate modeling is the nested nature of vehiclestop data. For example, ideally, the stop and citizen characteristics would be modeledat level one, officer characteristics modeled at level two, with level three encompassingthe community or organizational effects. In this manner, each lower level characteristicis nested within the higher level. The structure of the data may cause statisticalproblems for multivariate analyses because the assumption of uncorrelated errors maybe violated. Multilevel modeling, a specific type of multivariate analysis, corrects thisproblem by considering the hierarchical nature of the data (Raudenbush and Bryk,2002). Multilevel modeling is premised on the same assumptions as multivariatemodeling; however, hierarchical level modeling ensures that the coefficients for eachvariable are not biased by inappropriately grouping characteristics together acrosslevels of analysis. Thus, multilevel modeling is an advanced form of multivariateanalysis that offers a solution to one weakness of traditional multivariate analysis andis relevant for the analysis of police-citizen encounters.

A more recent addition to the vehicle stop outcome analysis toolbox is the outcometest. Originally formulated by Becker (1957) and subsequently applied to police searchesby Ayres (2001), the outcome test is based on the notion that if officers are profilingminority motorists, they will continue to search minorities even when the returns (i.e. thediscovery of contraband) are smaller for minorities than the returns gained whensearching white drivers (Anwar and Fang, 2006). Underlying this hypothesis is thepremise that the two groups of interest, the police and those carrying contraband, willmodify their behavior over time to maximize their desired outcome (Knowles et al., 2001;Ayres, 2001). Over the course of time, if no bias exists, a state of equilibrium will developin which the police search racial groups proportionately to their actual possession ofcontraband (Persico and Todd, 2008). At the point of equilibrium, it is only necessary tocompare the search success rates of different racial groups to assess if there is a disparitybetween these groups (Knowles et al., 2001; Persico and Todd, 2006, 2008). The search

PIJPSM33,1

success rate is calculated by dividing the number of searches in which officers seize sometype of contraband (e.g. drugs, illegal weapons) by the number of total searches (Fridell,2004, 2005; Ramirez et al., 2000). Any difference in the search success rates acrossrace/ethnicity groups indicates disparity.

The outcome test offers several advantages for analyzing vehicle stop outcomes(Engel, 2008). The data necessary for the analysis is usually accessible, provided thelaw enforcement agency collects information on discretionary searches, the race of thedriver, and the outcome of searches. The outcome test also provides an interpretableresult that does not require complex explanations. Finally, outcome test proponentssuggest that it avoids the concern of model misspecification once the state ofequilibrium is achieved (Knowles et al., 2001).

Notwithstanding its advantages, the outcome test has some weaknesses when appliedto vehicle stop outcomes (Engel and Tillyer, 2008). It is only appropriate for analysis ofdiscretionary searches (Fridell, 2004); thus, a large number of vehicle stop outcomes arenot considered in the analysis. The outcome test when applied to police searches alsoassumes that all racial groups carry contraband at equal rates (Antonovics and Knight,2004; Dharmapala and Ross, 2004; Hernandez-Murillo and Knowles, 2004), which has notbeen validated though empirical study. Further, when applied to police searches, theoutcome test assumes that police are only motivated by discovery of contraband;however, recent focus group research suggests that this is an untenable assumption, asofficers are motivated by a variety of factors beyond just the discovery of contraband,including officer safety and adherence to the principles of the law (Engel et al., 2007c).Finally, potentially the most troubling assumption of the outcome test when applied topolice searches is the reliance on a state of equilibrium, even though no independentmeasure of achieving equilibrium is included. For example, if search success rates areequivalent this could indicate either a police bias or that the equilibrium condition hasnot yet been met and further behavior alterations by both groups (i.e. officers and drugtraffickers) are necessary to reach equilibrium (Engel and Tillyer, 2008). Similar toconclusions drawn from multivariate analysis, no definitive conclusions can be drawnfrom the outcome test regarding racial bias when applied to police searches.

While multivariate analysis and the outcome test are the most common techniquesused to assess vehicle stop outcomes, other options are available. Propensity scoreanalysis has recently been applied to vehicle stop outcomes by comparing similarlysituated vehicle stops to determine if the outcomes in those stops are equivalent(Ridgeway, 2006). Propensity score analysis offers an easy to understand result anddoes not require the collection of additional information beyond the vehicle stop data.Put simply, propensity score analysis compares the vehicle stop outcomes of whitedrivers to minority drivers by matching the vehicle stops on all other factors except forthe driver’s race/ethnicity. In essence, the vehicle stops of the two groups (i.e. white andminority drivers) would be matched on all other potentially important variables relatedto the outcome (i.e. time of day, geographic location, reason for the stop, etc.) and then acomparison made between the groups to determine if there is a difference in outcomes.

One aspect of this technique that has not been thoroughly explored by social scienceresearchers is the need to weigh some vehicle stops more heavily than others in theanalysis, thereby potentially violating the assumption that all vehicle stops and theiroutcomes are equally important in the analysis. Moreover, similar to multivariatemodeling, propensity scores are susceptible to model misspecification; that is,

collection

information relevant to understanding vehicle stop outcomes may not be included,thereby biasing the result. To date, the only uses of propensity scores have been inCincinnati, Ohio (Riley et al., 2005; Ridgeway et al., 2006) and Oakland, California(Ridgeway, 2006). This technique has not been attempted for a state police agencywhose data may not be clustered enough to allow for propensity score matching.Further research is needed to assess the appropriateness of this technique.

Another tool for analyzing vehicle stop outcomes is Geographic Information Systems(GIS) analyses. This method graphs the data in space by creating a map to showdisparities in vehicle stop outcomes across race/ethnicity groups and/or geographicunits. This method also allows the vehicle stop data to be compared with othercommunity level data such as crime rates, economic indicators, etc. Various geographicanalysis techniques exist to examine the pattern of vehicle stop outcomes, such asidentifying the effects of spatial autocorrelation, conducting cluster analysis, calculatinga nearest neighbor index, and developing a spatial regression model. In addition,temporal analysis of the data can be computed within a GIS, which allows the data to beobserved, modeled, and analyzed across time periods. Some scholars suggest thatwithout considering the geographic location of the stop and its outcome, the analysesmay result in misleading findings (Fridell, 2004, 2005; McMahon et al., 2002). The use ofGIS does not supersede traditional statistical approaches to analyses; rather, it enhancesthese methods by including a spatial and temporal dimension. GIS is particularlyrelevant to state law enforcement agencies that patrol large geographic areas and need toprioritize resource allocation by providing information that is easy to interpret. Onelimitation of this tool is the requirement of a spatial locator for every vehicle stop.

A final option for examining vehicle stop outcomes is trend analysis, which graphsvehicle stops and their outcomes over time. In the case of police-citizen encounters,trend analysis can compare changes in vehicle stop outcome rates through the use ofthe binomial statistic. Graphing these trends can assist in determining if they areincreasing, decreasing or remaining stable. This tool is particularly important forpolice agencies that are interested in their organization’s behavior over time. Trendanalysis has not been frequently used to examine post-stop outcomes (for an exception,see Engel et al., 2007b), but has been routinely used to understand other criminal justicerelated phenomena over time. As the collection of data continues, there are morerigorous options for temporal analyses, such as the use of ARIMA, to identify if therates of vehicle stop outcomes for minority drivers have changed over time.

In summary, vehicle stop outcome analyses should include warnings, citations,arrests, searches, and the discovery of contraband. In addition, duration of the stop anduse of force may also be of interest. Multivariate analyses are recommended as a robustmethod to identify potential racial/ethnic disparities in vehicle stop outcomes, but thisapproach must consider the impact of specification error. In addition, the outcome test,despite its limitations, is a useful mechanism for identifying racial/ethnic disparities indiscretionary searches. Importantly, this technique is not appropriate fornon-discretionary police-citizen searches. Alternative methods of vehicle stop dataanalysis include propensity scores, GIS, and trend analysis. Each of these methods arerelatively new and require further examination to assess their utility in identifyingracial/ethnic disparities in vehicle stop outcomes.

PIJPSM33,1

4. ConclusionData collection and analysis of vehicle stop data has become commonplace in the last15 years (Fridell, 2005). Law enforcement agencies can now assess and begin tounderstand the decision-making process of their officers with the assistance of thesedata. The trend toward vehicle stop data collection across the nation offers severaladvantages for police agencies (Fridell et al., 2001). In particular, these efforts can assistin informing agencies about patterns and trends of disparities in the stops and stopoutcomes for specific racial/ethnic groups. In undertaking this self-evaluation, agenciesdemonstrate a commitment to unbiased policing, particularly in situations where anagency voluntarily initiates data collection or goes beyond what is legislatively orjudicially required of them. Moreover, understanding the patterns of vehicle stops andtheir outcomes can assist agencies in the effective and efficient allocation of resources,which are often prime considerations in the present budget-conscious environment.

Advantages of data collection and analysis must be weighed against the challengesof such an endeavor. Data collection may negatively affect officers’ morale,productivity, or workload. Officers are generally not supportive of data collectionefforts to assess their work behavior. Furthermore, these data collection efforts may beused against the agency by fueling lawsuits or changes in the laws regarding policebehavior. Data improperly analyzed can cause severe damage to individual officersand agencies that can take years to repair.

More importantly, data collection and analysis do not completely address the sourceand extent of bias in decision-making (Fridell et al., 2001). Simply collecting data onvehicle stops and their outcomes will not inform agencies about whether their officersare using race/ethnicity inappropriately in the decision making process. Data andmethodological limitations prevent social scientists from pinpointing the causal factorsleading to racial/ethnic disparities in vehicle stops. Racial and ethnic disparities invehicle stops could be the result of either bias or non-bias mechanisms(Tomaskovic-Devey et al., 2004).

None of the current data collection efforts tap into the underlying motivations formaking vehicle stops. Measurements of the factors that influence individual officerdecision-making are critical to determine whether or not officers are acting based onracial prejudice, animus, or cognitive bias. Thus, results suggesting that somedisparities exist for police stops, citations, searches, and arrests of differentracial/ethnic groups cannot be directly attributed to racially biased policing becausethe methods/analyses do not currently rule out alternative explanations of racialdisparities (Engel et al., 2002; McMahon et al., 2002). Without this information, it isimpossible to conclusively determine that racial/ethnic disparity in vehicle stops oroutcomes is evidence of discrimination.

Future research may have to advance beyond quantitative analyses and explorequalitative studies to address the underlying motivations and reasons for officerdecision making. This alternative approach to studying the existence and extent ofbias based policing likely will require asking officers to describe their decision makingprocess through the use of interviews and/or focus groups. Initial research in this areasuggests that there is rich, detailed information to be gathered from officers regardingtheir decision making using such methodologies (Engel et al., 2007c).

The importance of identifying and understanding any disparities experienced byminorities when interacting with the police will continue to present social, cultural, and

collection

scientific challenges as the twenty-first century unfolds. The recent spike in vehiclestop data collection and analysis is evidence that society is still grappling withensuring that minority citizens are treated in a fair and equitable manner by lawenforcement officials. The responsibility of social scientists is to ensure that thecollection and analysis of vehicle stop data is built on best practices. Reliance on bestpractices and innovation derived from these principles offers the greatest chance ofdeveloping a more robust understanding of racial/ethnic disparities in vehicle stopsand vehicle stop outcomes.

1. Wilkins v. Maryland State Police et al., Civ. No. MJG-93-468 (D.Md., 1993).

2. State of New Jersey v. Soto et al., 734 A. 2d 350 (N.J. Super. 1996).

3. A comprehensive review of all state agencies was conducted and culled to form the basis ofthe recommendations offered in this discussion.

4. State police agencies are the focus of this discussion; however, many of the issues raised inregard to data collection and analysis at the state level are also applicable to local agencies.

5. In Florida, legislation requires the department to develop an anti-racial profiling policy. InLouisiana, if departments have a racial profiling policy, no data collection is required. InOregon, legislation encourages and funds data collection.

6. For example, X-Y coordinates of the vehicle stop, a specific street address or highwaymarker for the stop and/or the county, municipality, or police jurisdiction is recommended.

7. The reason for the stop should include categories such as speeding, equipment inspections,pre-existing information, and registration or license violations.

8. These dimensions simply indicate the risk of being stopped and are not organized withinlegal or extra-legal categories.

9. A study conducted in North Carolina used both not-at-fault and at-fault crash data to formtheir benchmark (for a complete description, see Smith et al., 2003).

10. One underlying assumption of this approach is that the registered owner of the vehiclephotographed was driving at the time of the photograph.

11. These approaches are not mutually exclusive as information can be simultaneously collectedon all passing vehicles and only those vehicles violating the law.

12. Chavez v. Illinois State Police, 310 F.3d (7th Cir. 2001).

13. State of New Jersey v. Kennedy, 588 A. 2d 834 (1991).

14. State of New Jersey v. Smith, 703 A. 2d 954 (1997).

15. US v. Alcaraz-Arellano, 302 F. Supp. 2d 1217 (2004).

16. US v. Lindsey, 288 F. Supp. 2d 1196 (2003).

17. US v. Mesa-Roche, 288 F. Supp. 2d 1172 (2003).

References

Alpert, G.P., Smith, M.R. and Dunham, R.G. (2004), “Toward a better benchmark: assessing theutility of not-at-fault vehicle crash data in racial profiling research”, Justice Research andPolicy, Vol. 1, pp. 43-69.

PIJPSM33,1

Alpert, G.P., Becker, E., Gustafson, M.A., Meister, A.P., Smith, M.R. and Strombom, B.A. (2006),“Pedestrian and motor vehicle post-stop data analysis report”, Analysis Group, LosAngeles, CA, available at: www.analysisgroup.com/AnalysisGroup/uploadedFiles/Publishing/Articles/LAPD_Data_Analysis_Report_07-5-06.pdf (accessed August 23, 2008).

Alpert Group (2004), “Miami-Dade Police Department racial profiling study”, available at: www.aclufl.org/pdfs/MiamiDade%20Police%20Department%20%20Racial%20Profiling%20Study%20%20The%20Alpert%20Group.pdf (accessed August 17, 2008).

Antonovics, K. and Knight, B. (2004), “A new look at racial profiling: evidence from the BostonPolice Department”, Working Paper No. 10634, National Bureau of Economic Research,Cambridge, MA.

Anwar, S. and Fang, H. (2006), “An alternative test of racial profiling in motor vehicle searches:theory and evidence”, The American Economic Review, Vol. 96, pp. 127-51.

Ayres, I. (2001), Pervasive Prejudice? Unconventional Evidence of Racial and GenderDiscrimination, University of Chicago Press, Chicago, IL.

Batton, C. and Kadleck, C. (2004), “Theoretical and methodological issues in racial profilingresearch”, Police Quarterly, Vol. 7, pp. 30-64.

Becker, G.S. (1957), The Economics of Discrimination, University of Chicago Press, Chicago, IL.

Buerger, M.E. and Farrell, A. (2002), “The evidence of racial profiling: interpreting documentedand unofficial sources”, Police Quarterly, Vol. 5, pp. 272-305.

Davis, R.L. (2001), Racial Profiling: What Does the Data Mean?, National Organization of BlackLaw Enforcement Executives, Alexandria, VA.

Davis, R.L., Gillis, I. and Foster, M. (2001), Perspective: Racial Profiling – A Symptom ofBias-based Policing, National Organization of Black Law Enforcement Executives,Alexandria, VA.

Dharmapala, D. and Ross, S.L. (2004), “Racial bias in motor vehicle searches: additional theoryand evidence”, Contributions to Economic Analysis & Policy, Vol. 3, pp. 1-21.

Durose, M.R., Smith, E.L. and Langan, P.A. (2007), Contacts between Police and the Public, 2005,Bureau of Justice Statistics, US Department of Justice, Washington, DC.

Engel, R.S. (2008), “A critique of the ‘outcome test’ in racial profiling research”, Justice Quarterly,Vol. 25, pp. 1-36.

Engel, R.S. (2005), “Citizens’ perceptions of procedural and distributive injustice during trafficstops with police”, Journal of Research in Crime and Delinquency, Vol. 42, pp. 445-81.

Engel, R.S. and Calnon, J.M. (2004), “Comparing benchmark methodologies for police-citizencontacts: traffic stop data collection for the Pennsylvania State Police”, Police Quarterly,Vol. 7, pp. 97-125.

Engel, R.S. and Tillyer, R. (2008), “Searching for equilibrium: the tenuous nature of the outcometest”, Justice Quarterly, Vol. 25, pp. 54-71.

Engel, R.S., Calnon, J.M. and Bernard, T.J. (2002), “Theory and racial profiling: shortcomings andfuture directions in research”, Justice Quarterly, Vol. 19, pp. 249-73.

Engel, R.S., Cherkauskas, J.C. and Tillyer, R. (2007a), “Traffic stop data analysis study report:final literature review and review of other jurisdictions”, University of Cincinnati PolicingInstitute, Cincinnati, OH.

Engel, R.S., Klahm, C.F. and Tillyer, R. (2010), “Citizens’ demeanor, race, and traffic stops”, inRice, S.K. and White, M. (Eds), Race, Ethnicity & Policing: The Issues, Methods, Researchand Future, New York University Press, New York, NY.

collection

Engel, R.S., Tillyer, R. and Cherkauskas, J.C. (2007c), Understanding Best Search and SeizurePractices: Final Report, University of Cincinnati Policing Institute, Cincinnati, OH.

Engel, R.S., Calnon, J.M., Liu, L. and Johnson, R.R. (2004), Project on Police-citizen Contacts, Year1 Final Report, University of Cincinnati, Cincinnati, OH.

Engel, R.S., Frank, J., Tillyer, R. and Klahm, C. (2006), Cleveland Division of Police Traffic StopStudy: Final Report, University of Cincinnati Policing Institute, Cincinnati, OH.

Engel, R.S., Tillyer, R., Cherkauskas, J.C. and Frank, J. (2007b), Traffic Stop Data Analysis Study:Year 1 Final Report, University of Cincinnati Policing Institute, Cincinnati, OH.

Engel, R.S., Tillyer, R., Stoddard, C. and Johnson, R. (2007d), Project on Police-citizen Contacts:Year 3 and 4 Final Report, January 2004-December 2005, University of CincinnatiPolicing Institute, Cincinnati, OH.

Engel, R.S., Calnon, J.M., Tillyer, R., Johnson, R.R., Liu, L. and Wang, X. (2005), Project onPolice-citizen Contacts, Year 2 Final Report, University of Cincinnati, Cincinnati, OH.

Farrell, A., McDevitt, J., Cronin, S. and Pierce, E. (2003), Rhode Island Traffic Stop Statistics ActFinal Report, Northeastern University Institute on Race and Justice, Boston, MA.

Farrell, A., McDevitt, J., Bailey, L., Andresen, C. and Pierce, E. (2004), “Massachusetts racial andgender profiling study”, Northeastern University Institute on Race and Justice, Boston,MA, available at: www.racialprofilinganalysis.neu.edu/IRJsite_docs/finalreport.pdf(accessed August 14, 2008).

Fridell, L. (2004), By the Numbers: A Guide for Analyzing Race Data from Vehicle Stops, PoliceExecutive Research Forum, Washington, DC.

Fridell, L. (2005), Understanding Race Data from Vehicle Stops: A Stakeholder’s Guide, PoliceExecutive Research Forum, Washington, DC.

Fridell, L., Lunney, R., Diamond, D. and Kubu, B. (2001), Racially Biased Policing: A PrincipledResponse, Police Executive Research Forum, Washington, DC.

General Accounting Office (2000), “Racial profiling: limited data available on motorist stops”,GAO/GGD-00-41, General Accounting Office, Washington, DC.

Grogger, J. and Ridgeway, G. (2006), “Testing for racial profiling in traffic stops from behind aveil of darkness”, Journal of the American Statistical Association, Vol. 101, pp. 878-87.

Gumbhir, V.K. (2004), “Final report on the Eugene Police Department’s traffic stop data(2002-2003)”, condensed report, University of Oregon, Eugene, OR.

Hanushek, E.A. and Jackson, J.E. (1977), Statistical Methods for Social Scientists, Academic Press,Orlando, FL.

Harris, D.A. (2002), Profiles in Injustice: Why Racial Profiling Cannot Work, New Press, NewYork, NY.

Harris, D.A. (2006), “US experiences with racial and ethnic profiling: history, current issues, andthe future”, Critical Criminology, Vol. 14, pp. 213-39.

Hernandez-Murillo, R. and Knowles, J. (2004), “Racial profiling or racist policing? Bounds tests inaggregate data”, International Economic Review, Vol. 45, pp. 959-89.

Hickman, M.J. (2005), “Traffic stop data collection policies for State Police, 2004”, NCJ 209156,Bureau of Justice Statistics, US Department of Justice, Washington, DC.

Ingram, J.R. (2007), “The effect of neighborhood characteristics on traffic citation practices of thepolice”, Police Quarterly, Vol. 10, pp. 371-93.

Institute on Race and Poverty (2001), Saint Paul Traffic Stop Data Analysis, University ofMinnesota, Minneapolis, MN.

PIJPSM33,1

Knowles, J., Persico, N. and Todd, P. (2001), “Racial bias in motor vehicle searches: theory andevidence”, Journal of Political Economy, Vol. 109, pp. 203-29.

Lamberth, J. (1994), “Revised statistical analysis of the incidence of police stops and arrests ofblack drivers/travelers on the New Jersey Turnpike between exits or interchanges 1 and 3from the years 1988 through 1991”, Temple University, Philadelphia, PA.

Lamberth, J. (1996), “A report to the ACLU”, American Civil Liberties Union, New York, NY.

Lamberth, J. (2003), “Racial profiling study and services: a multijurisdictional assessment oftraffic enforcement and data collection in Kansas”, available at: www.racialprofilinganalysis.neu.edu/IRJ_docs/KS_2003.pdf. (accessed August 22, 2008).

Lange, J.E., Johnson, M.B. and Voas, R.B. (2005), “Testing the racial profiling hypothesis orseemingly disparate vehicle stops on the New Jersey Turnpike”, Justice Quarterly, Vol. 22,pp. 193-223.

Lovrich, N.P., Gaffney, M.J., Mosher, C.M., Pickerill, M. and Smith, M.R. (2003), “WSP traffic stopdata analysis project: data analysis project report”, University of Washington, Olympia,WA.

Lovrich, N.P., Gaffney, M.J., Mosher, C.M., Pickerill, M. and Pratt, T.C. (2005), “Analysis of trafficstop data collected by the Washington State Patrol: assessment of racial and ethnic equityand bias in stops, citations, and searches using multivariate quantitative andmulti-method qualitative research techniques”, University of Washington, Olympia, WA.

Lundman, R.J. (1996), “Demeanor and arrest: additional evidence from previously unpublisheddata”, Journal of Research in Crime and Delinquency, Vol. 33, pp. 306-23.

Lundman, R.J. (1994), “Demeanor or crime? The Midwest city police-citizen encounters study”,Criminology, Vol. 32, pp. 631-56.

Lundman, R.J. and Kaufman, R.L. (2003), “Driving while Black: effects of race, ethnicity, andgender on citizen self-reports of traffic stops and police actions”, Criminology, Vol. 41,pp. 195-220.

McMahon, J., Garner, J., Davis, R. and Kraus, A. (2002), “How to correctly collect and analyzeracial profiling data: your reputation depends on it!”, Government Printing Office,Washington, DC.

Meehan, A.J. and Ponder, M.C. (2002a), “Race and place: the ecology of racial profiling AfricanAmerican motorists”, Justice Quarterly, Vol. 19, pp. 399-430.

Meehan, A.J. and Ponder, M.C. (2002b), “How roadway composition matters in analyzing policedata on racial profiling”, Police Quarterly, Vol. 5, pp. 306-33.

National Conference of State Legislatures (2001), State Laws Address “Racial Profiling”, NationalConference of State Legislatures, Washington, DC.

Novak, K. (2004), “Disparity and racial profiling in traffic enforcement”, Police Quarterly, Vol. 7,pp. 65-96.

Persico, N. and Todd, P. (2008), “The hit rates test for racial bias in motor-vehicle searches”,Justice Quarterly, Vol. 25, pp. 37-53.

Persico, N. and Todd, P. (2006), “Using hit rates to test for racial bias in law enforcement: vehiclestops in Wichita”, The Economic Journal, Vol. 116, pp. F351-67.

Ramirez, D., McDevitt, J. and Farrell, A. (2000), A Resource Guide on Racial Profiling DataCollection Systems: Promising Practices and Lessons Learned, US Department of Justice,Washington, DC.

Raudenbush, S.W. and Bryk, A.S. (2002), Hierarchical Linear Models, 2nd ed., Sage Publications,Newbury Park, CA.

collection

Ridgeway, G. (2006), “Assessing the effect of race bias in post-vehicle stop outcomes usingpropensity scores”, Journal of Quantitative Criminology, Vol. 22, pp. 1-29.

Ridgeway, G., Schell, T., Riley, K.J., Turner, S. and Dixon, T.L. (2006), Police-community relationsin Cincinnati: Year Two Evaluation Report, RAND Corporation, Santa Monica, CA.

Riley, K.J., Turner, S., MacDonald, J., Ridgeway, G., Schell, T., Wilson, J., Dixon, T.L., Fain, T. andBarnes-Proby, D. (2005), Police-community Relations in Cincinnati, RAND Corporation,Santa Monica, CA.

Rojek, J., Rosenfeld, R. and Decker, S. (2004), “The influence of driver’s race on traffic stops inMissouri”, Police Quarterly, Vol. 7, pp. 126-47.

Schafer, J.A., Carter, D.L., Katz-Bannister, A.J. and Wells, W.M. (2006), “Decision making intraffic stop encounters: a multivariate analysis of police behavior”, Police Quarterly, Vol. 9,pp. 184-209.

Schmitt, E.L., Langan, P.A. and Durose, M.R. (2002), “Characteristics of drivers stopped bypolice, 1999”, US Department of Justice, Washington, DC.

Smith, E.L. and Durose, M.R. (2006), “Characteristics of drivers stopped by police, 2002”, USDepartment of Justice, Washington, DC.

Smith, M.R. and Alpert, G.P. (2002), “Searching for direction: courts, social science, and theadjudication of racial profiling claims”, Justice Quarterly, Vol. 19, pp. 673-703.

Smith, W.R., Tomaskovic-Devey, D., Zingraff, M.T., Mason, H.M., Warren, P.Y. and Wright, C.P.(2003), The North Carolina Highway Traffic Study: Final Report to the National Institute ofJustice, US Department of Justice, Washington, DC.

Solop, F.I. (2004a), “Comparative statistical analysis of I-17 stop data and I-17 violator data:Yavapai County, Arizona”, unpublished manuscript, Flagstaff, AZ.

Solop, F.I. (2004b), “Statistical analysis of I-40 stop data and I-40 violator data: Coconino County,Arizona”, unpublished manuscript, Flagstaff, AZ.

Strom, K.J. and Durose, M.R. (2000), “Traffic stop data collection policies for state police, 1999”,NCJ 180776, Bureau of Justice Statistics, US Department of Justice, Washington, DC.

Strom, K.J., Brien, P. and Smith, S. (2001), “Traffic stop data collection policies for state police,2001”, NCJ 191158, Bureau of Justice Statistics, US Department of Justice, Washington, DC.

Tillyer, R., Engel, R.S. and Wooldredge, J. (2008), “The intersection of racial profiling and thelaw”, Journal of Criminal Justice, Vol. 36 No. 2, pp. 138-53.

Tomaskovic-Devey, D., Mason, H.M. and Zingraff, M. (2004), “Looking for the driving whileBlack phenomena: conceptualizing racial bias processes and their associateddistributions”, Police Quarterly, Vol. 7, pp. 3-29.

Walker, S. (2001), “Searching for the denominator: problems with police traffic stop data and anearly warning system solution”, Justice Research and Policy, Vol. 3, pp. 63-95.

Withrow, B.L. (2004), “Race-based policing: a descriptive analysis of the Wichita stop study”,Police Practice and Research, Vol. 5, pp. 223-40.

Worden, R. and Shepard, R. (1996), “Demeanor, crime, and police behavior: a reexamination ofthe police services study data”, Criminology, Vol. 34, pp. 83-105.

Corresponding authorRob Tillyer can be contacted at: rob.tillyer@utsa.edu.

PIJPSM33,1

To purchase reprints of this article please e-mail: reprints@emeraldinsight.comOr visit our web site for further details: www.emeraldinsight.com/reprints

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Best practices in vehicle stop data collection and analysis

Documents

Transcript of Best practices in vehicle stop data collection and analysis

SHOPPERS STOP - NSE

BEST PRAC BEST PRACTICES - Sophia College, Ajmer

Stop location design

Lebanon Bus Stop Design Study

One-stop Lifestyle Platform - HKEX :: HKEXnews

We Won't Stop Dreaming - Commack Schools

THE STOP SIX CHOICE NEIGHBORHOOD ...

MAIN VEHICLE SPECIFICATIONS

Stop feeling: inhibition of emotional interference following stop-signal trials

BEST PRACTICE

40 Ingrijire post stop cardiorespirator (1)

Lateral Vehicle Dynamics Control and Vehicle State Estimation

Heikin Ashi - The Stop Hunter

'One of the best theatres in Australia. Full stop.'

WOULD STOP EMIGRANT RECRUITING - eVols

EMB-STOP - ICP Wind

HOW TO STOP PROCRASTINATION

power stop - Canal da Peça

DON'T LET OBSOLESCENCE STOP YOUR PRODUCTION

oil stop valve (osv) • specification sheet