ADVICE FOR THE COURTS? SUFFICIENTLY RELIABLE ASSISTANCE WITH FORENSIC SCIENCE Advice for the courts?...

ADVICE FOR THE COURTS?SUFFICIENTLY RELIABLE ASSISTANCE WITH FORENSIC SCIENCE

Advice for the courts?

Sufficiently reliable

assistance with forensic

science and medicine

(Part 2)

By Gary Edmond*

Professor and Director, Expertise, Evidence & Law Program, School

of Law, The University of New South Wales

Abstract In recent years, following public inquiries (for example, the GoudgeInquiry, 2007–08), reviews (for example, the US National Academy of Science,2009; the Law Commission of England and Wales, 2011), systematic analysis ofwrongful convictions (for example, Innocence Projects) and empirical studies,weaknesses with many types of forensic science and the frailty of the adversarialcriminal trial have been exposed, though inadequately addressed. Drawingupon emerging empirical evidence from a variety of common law jurisdictions(and recent work in the sociology of science) and building upon the discussion ofthe Law Commission’s Report and draft Bill (in Part 1), this article considers onemeans of helping common law courts to respond to some of the primarydifficulties raised by incriminating forensic science and forensic medicineevidence. The proposal, involving an independent multidisciplinary advisory panel(or MAP) reviewing impugned forensic science and medical techniques to assesstheir reliability in order to provide support with admissibility determinations,is intended to provide practical assistance with controversial expert opinion

doi:10.1350/ijep.2012.16.3.405

THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF (2012) 16 E&P 263–297 263

* Email: [email protected]. This research was supported by the ARC (FT0992041 andLP120100063). I would like to thank Ron Allen, Simon Cole, David Hamer, Brynn Hibbert, SheilaJasanoff, Trudo Lemmens, Andrew Ligertwood, Peter McClellan, Jenny McEwan, David Mercer,Glenn Porter, Mike Redmayne, Mehera San Roque, Kent Roach, Andy Roberts, Paul Roberts, SimonStern, Wendy Wagner and Tony Ward for critical contributions. Earlier versions were presented atthe John F. Kennedy School of Government, Harvard University, the School of Law, University ofToronto, the Society for the Social Studies of Science (4S) annual conference, Cleveland, 2011, theAustralian Academy of Forensic Sciences, and the Expressions Workshop, 2012.

evidence adduced by the state though without excessive encroachment uponthe traditional party-dominated accusatorial trial.

Keywords Expert; Panel; Advisory body; Admissibility; Law reform; Law andscience; Court-appointed expert; Daubert; Impartial

1. Introduction to some socio-legal problems (with expertise)

hose working in the accusatorial trial tradition have not been suffi-ciently attentive to the reliability of forensic science evidence. States,through their police, forensic scientists, prosecutors and judges, have

been producing, adducing and admitting, and tribunals of fact have beenroutinely relying upon, incriminating expert opinion evidence that is eitherunreliable or of unknown reliability. Until relatively recently, most of thisevidence went substantially unchallenged, despite its questionable epistemicprovenance, the fact that it could have been empirically assessed using fairlystandardised research methods, and notwithstanding express legal commitmentto truth (i.e. rectitude of decision), fairness and the need to avoid convicting theinnocent.1 Of significance, this failure to challenge and exclude unreliable expertopinion evidence and expert opinion of unknown reliability was not restricted tojurisdictions without reliability-based admissibility standards.2

In the common law tradition, the trial and appellate judges responsible for theadmission and review of incriminating expert opinion evidence are almostentirely dependent on the parties for the evidence brought before them.3 The mainproblems for accusatorial criminal justice systems, given the dependence on inter-ested parties of markedly different resources and abilities, are how courts (andfact-finders) obtain information about the representativeness, quality and,ultimately, probative value of techniques and derivative opinion, and how shouldscientific, technical and experiential knowledge claims be presented andcontested before, and evaluated by, lay participants?4

264 THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF

ADVICE FOR THE COURTS? SUFFICIENTLY RELIABLE ASSISTANCE WITH FORENSIC SCIENCE

T

1 Many past convictions have relied (at least in part) upon expert opinions derived using unreliabletechniques or opinions expressed in terms that are difficult to reconcile with what was knownempirically.

2 G. Edmond, ‘Is Reliability Sufficient? The Law Commission and Expert Evidence in Internationaland Interdisciplinary Perspective (Part 1)’ (2012) 16 E&P 30 (hereafter ‘Part 1’).

3 There may be scope at common law, or provision under statute, for court-appointed experts and/orspecial masters, but they are very rarely engaged. For a recent indication of reticent responses toexperimental psychological research and some of the limits on current approaches to taking‘judicial notice’, see Aytugrul v The Queen [2012] HCA 15.

4 This article is primarily oriented to the first question. Some issues relating to the second questionare discussed in ‘Impressions and Expressions’ (2012) 44 Australian Journal of Forensic Sciences(forthcoming) and (2012) 10 Law, Probability & Risk (forthcoming).

For a variety of reasons parties to litigation are not always capable of providingrelevant, let alone balanced and elucidatory, information about the reliability ofexpert opinion evidence. There is, in addition, a tendency for lawyers and judgessteeped in the common law tradition to focus on legal authority, past practice andearlier interpretations of rules, to the detriment of technical insights and scien-tific knowledge. In combination with the common law’s emphasis on theindividual case, these factors have contributed to the development of practicesthat are inconsistent across similar types of evidence and issues.5 In criminalproceedings, relevant scientific and technical studies—what I will describe asexogenous knowledges—and standardised methodological techniques are infre-quently invoked (or cited) by parties, expert witnesses and judges, and even lessfrequently discussed. Admissibility decision-making is often based upon opinionsand impressions that are neither supported by research nor publication. Too oftenthe evidence provided by forensic scientists is developed without standardisedmethods, without empirical testing and, often, with indifference to method-ological and institutional problems (for example, the way to manage cognitivebiases or express opinions) notorious among research scientists. Compoundingthese tendencies, and dampening internal interest in reform, judges tend tomaintain high levels of confidence in the efficacy of their common law processes,particularly trial safeguards (to promote accuracy and fairness) and the ability ofjurors and appellate courts to assess and review rationally expert opinions (ofunknown probative value).6

Given these and other limitations, this article presents a proposal to enhance themanner in which courts respond to expert opinion evidence, particularly incrimi-nating expert opinion evidence, at the admissibility stage. That is, where judgesdetermine whether contested evidence should be admitted to the trial for consid-eration by the tribunal of fact—often a jury. Unlike many other responses toperceived problems with expertise, this proposal aims to avoid excessive inter-ference with accusatorial trial practice while providing assistance with theapplication of admissibility rules in ways that are consistent with criminal justiceprinciples and aspirations. It represents an attempt to improve practice largelywithin existing common law frameworks.7 In addition, the proposal is developed

THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF 265


5 Approaches to the same kinds of evidence across jurisdictions and to similar kinds of evidencewithin jurisdictions seem to be incoherent, see G. Edmond, S. Cole, E. Cunliffe (and A. Roberts),‘Admissibility Compared’, paper presented at Law & Society Conference, San Francisco, 2011.

6 See Part 1; G. Edmond and A. Roberts, ‘Procedural Fairness, the Criminal Trial and Forensic Scienceand Medicine’ (2011) 33 Sydney L Rev 359; G. Edmond and M. San Roque, ‘The Cool Crucible:Forensic Science Evidence and the Frailty of the Criminal Trial’ (2012) 23 Current Issues in CriminalJustice (forthcoming).

7 It is not intended as a defence of these frameworks, or associated traditions and assumptions.

with sensitivity to contemporary work in the sociology of expertise andtechnology assessment.8 The proposal does not, in consequence, attempt to definescience, locate impartial expertise, identify some proper extra-legal value of scien-tific and technical evidence, address problems through judicial education,remove public participation from the accusatorial trial, or diminish the ability tocontest the meaning and value of admissible expert opinion evidence (andexpertise) in the public setting of the trial (and appeal).9 More modestly, it aims toassist courts with some admissibility determinations in order to improvefact-finding at trial and enhance the accuracy, fairness and efficiency of criminaljustice processes and outcomes. By assisting with assessments of reliability inadmissibility decision-making the proposal seeks to improve institutionalefficiency, consistency in the treatment of expert opinion evidence and theaccuracy of verdicts.

In response to emerging problems and the limitations of current practice (seePart 1), the proposal entails establishing a multidisciplinary advisory panel (MAP).Composed of independent experts, the panel would undertake a pragmatic—i.e.practical, in the sense of expedited and restricted—assessment of the reliabilityof some forensic science or medical technique(s) used or proposed for use incriminal proceedings. Based on a compressed review and synthesis of thepublished literature, and privileging experiment and empirical studies, theassessment is intended to provide grounded information about the validity andreliability of a technique (or suite of techniques). The panel’s published advice isintended to facilitate, and perhaps presupposes, a degree of deference from thecourts—what I describe as principled deference.10 Moreover, the panel does noteliminate lay participation in criminal justice practice, although it may modifythe terms of engagement with some types of expert opinion evidence—namely



8 See, e.g., D. Collingridge and C. Reeve, Science Speaks to Power: The Role of Experts in Policy Making (StMartin’s Press: New York, 1986); A. Rip, T. Misa and J. Schot (eds.), Managing Technology in Society: TheApproach of Constructive Technology Assessment (Pinter: London/NY, 1995); A. Irwin, ‘STS Perspectiveson Scientific Governance’ in E. Hackett, O. Amsterdamska, M. Lynch and J. Wajcman (eds.), TheHandbook of Science and Technology Studies (MIT Press: Cambridge, MA, 2008) 583.

9 See R. Smith and B. Wynne, Expert Evidence (Routledge: London, 1989) and S. Jasanoff, Science at theBar (Harvard University Press: Cambridge, MA, 1995).

10 There may be a degree of deference at the admissibility stage, although reliability can be debatedduring any admissibility hearing and admissibility should be based on the judge being persuadedabout the value of the technique and testimony based on some kind of empirical evidence. Where,however, expert opinion evidence is deemed admissible, the accusatorial trial is unambiguouslypredicated upon the jury being in a position to actually understand and assess all of the evidence.See R. Allen and J. Miller, ‘The Common Law Theory of Experts: Deference or Education’ (1993) 87Nw U L Rev 1131; R. Allen, ‘Expertise and the Daubert Decision’ (1994) 84 J Crim L & Criminology1157.

those that appear to lack empirical support and are therefore insufficiently reliableto admit.11 The panel is intended to ease the burden on judges and lawyers (andjuries), to produce greater consistency across criminal proceedings, and to limitthe imposition of complex decisions about reliability on publics that are neitherresourced nor, given their dependent and largely passive role in the accusatorialtrial, in a position to assess the underlying reliability of techniques, interpreta-tions and investigative experience.

Drawing on legal principle, this article assumes that forensic science techniquesand derivative opinions should have demonstrable support for their reliability—inthe sense of trustworthiness—before they are admitted and relied upon to supportproof of guilt.12 That is, we should have some basis for believing that an expert canactually do what he or she claims. In many cases this will require studies thatdemonstrate the validity of techniques, along with some indication of the rate oferror (or proficiency of practitioners). Such questions, and their empiricalwarrant, will focus the attention of the panel.

The following discussion is limited to accusatorial proceedings in common lawjurisdictions. While there are significant differences—in legal traditions, rules,procedures and practices, as well as the appointment of judges and theresourcing of courts and lawyers—across the common law world, many of theissues considered in this article are, to a considerable degree, shared. Perhapsmore significant than formal differences are similarities in actual practices;particularly in the convergence toward reliability standards and the kinds offorensic science and medical evidence that are, in apparent violation of suchstandards (and broader criminal justice principles and aspirations), routinelyadmitted.

2. Legal values and objectives: A quick overview of accusatorial trial

principles

To begin, it is important to introduce what are generally considered to be thedominant principles and objectives underlying the accusatorial criminal trial inrelation to proof. The primary concern in the ‘rational tradition’ is with factual



11 I use ‘insufficiently reliable’—a term from the Law Commission’s draft Bill—synonymously with‘not demonstrably reliable’. However, see Part 1, Section 3.

12 It is not my intention to unpack ‘reliability’. There are several useful expositions, but reliability inadmissibility jurisprudence is primarily oriented to the quality, value and trustworthiness ofevidence rather than the narrow technical meaning(s) of reliability frequently employed byscientists. See the discussion in Daubert v Merrell Dow Pharmaceuticals Inc., 509 US 579 (1993) andKumho Tire Co. v Carmichael, 526 US 137 at 147 (1999).

rectitude (sometimes characterised as truth).13 Almost all theorists, commentatorsand practitioners (including judges) emphasise the fundamental importance of acorrect decision—that is, convicting only those who are in fact guilty of anoffence.14 In consequence, the burden and standard of proof at trial are arrangedso that risks (should) lie mainly with the accuser (i.e. the state). In principle thesystem ought to err on the side of caution, with those accused obtaining thebenefit of uncertainty.15 The state, therefore, is obliged to prove every element ofan offence beyond reasonable doubt. Conversely, the accused is presumedinnocent and, in most circumstances, has no obligation to do (or say) anything inhis or her defence.16

There are additional and sometimes competing considerations associated withprosecution and proof; such as concern with the fairness of proceedings. Theaccusatorial trial incorporates a variety of mechanisms that purport to operate assafeguards (or protections) for the parties—especially the accused. Those accusedand prosecuted have basic entitlements to: know the case against them; accesslegal advice; inspect and challenge evidence; attend proceedings; confrontwitnesses through cross-examination; call witnesses and adduce evidence; havesome types of unfairly prejudicial evidence excluded, and so on.17 The tinyproportion of cases that go to trial are heard before juries composed of eligible



13 W. Twining, ‘The Rationalist Tradition of Evidence Scholarship’ in Rethinking Evidence: ExploratoryEssays (Blackwell: London, 1990); J. Jackson, ‘Two Methods of Proof in Criminal Procedure’ (1988) 51MLR 249; P. Roberts and A. Zuckerman, Criminal Evidence, 2nd edn (Oxford University Press: Oxford,2010); M. Redmayne, Expert Evidence and Criminal Justice (Oxford University Press: Oxford, 2001); A.Duff, L. Farmer, S. Marshall and V. Tadros, The Trial on Trial, 3 vols (Hart: Oxford, 2004); J. Jackson andS. Summers, The Internationalisation of Criminal Evidence: Beyond the Common Law and Civil LawTraditions (Cambridge University Press: Cambridge, 2012). In the Anglo-American tradition,consider the works of Bentham, Thayer and Wigmore, discussed in W. Twining, Theories of Evidence:Bentham and Wigmore (Weidenfeld and Nicolson: London, 1985).

14 Such commitments are sometimes embedded in rules and statutes (e.g. US FRE 102) and influentialappeals to superior courts (e.g. Daubert v Merrell Dow Pharmaceuticals Inc., 509 US 579 (1993), KumhoTire Co. v Carmichael, 526 US 137 (1999) and R v Nikolovski [1996] 3 SCR 1197 at [13]). US Federal Rules ofEvidence (FRE) 102 states: ‘These rules should be construed so as to administer every proceedingfairly, eliminate unjustifiable expense and delay, and promote the development of evidence law,to the end of ascertaining the truth and securing a just determination.’ See also the CriminalProcedure Rules 2011 (SI 2011 No. 1709), r. 1.1 (England and Wales) and the establishment of theCriminal Cases Review Committee to identify substantial mistakes.

15 While the system is especially concerned to avoid wrongful convictions, enhancing the quality ofexpert evidence should simultaneously reduce the number of false acquittals and help to preventmisguided investigations and prosecutions that enable actual perpetrators to reoffend.

16 Historically, these values have been embodied in ideas such as it is better to let guilty persons gofree (with numbers varying between 5 and 1,000) than convict (and sometimes execute) oneinnocent person, see A. Volokh, ‘n Guilty Men’ (1997) 146 U Penn Law Rev 173.

17 Some of these traditional entitlements (or ‘rights’) have been reinterpreted—often read down—inthe wake of 9/11 and terrorist trials in many adversarial jurisdictions.

(though not always representative) citizens (or judges), and those convictedusually have some right to appeal against conviction based on alleged proceduralerrors, unfairness, or a mistaken (or ‘unsafe’) verdict.18 In most jurisdictions,though sometimes at the prerogative of the Executive, there is scope for furtherreview of convictions even after entitlements to appeals have been exhausted—such as where ‘fresh’ evidence emerges.19

Interest in accuracy and fairness also manifest in concerns about the value anduses of some types of evidence and even whether adversarial procedures—incombination with largely unanswered questions about the abilities of the tribunalof fact—facilitate an adequate assessment of evidence (see Section 7 below). Thereare many adjectival rules either proscribing or restricting the admission and useof certain types of evidence. Limiting the use of hearsay, preventing (or restricting)reliance on past misconduct to draw inferences about behaviour, along withproscriptions on many types of opinion, are all examples of such provisions. Therules governing evidence and its uses are often described as admissibility (orexclusionary) rules.

In recent decades courts have generally become more receptive to evidence, withmany strict exclusionary rules tempered to facilitate admission. Some have beenmodified by statute, others have been modified by judges, in ways that embodyemerging socio-institutional concerns about system efficiencies and the need tomanage diminishing resources better.20 Simultaneously, most senior judgesrehearse long-standing confidence in the jury, and especially modern jurors.Consequently, judges and legislatures across the Anglophone world have beenrelaxing admissibility rules in criminal proceedings and seem to be increasinglyreluctant to exclude apparently relevant incriminating evidence.21 In most juris-dictions there has been a gradual movement toward ‘free proof’ as rules that wereonce interpreted in ways that kept information from the tribunal of fact—such as



18 On the decline of the trial and juries, see respectively: R. Burns, The Death of the American Trial(University of Chicago Press: Chicago, IL, 2009) and N. Vidmar and V. Hans, American Juries: TheVerdict (Prometheus: Amherst, NY, 2007).

19 Concerns about post-conviction review and miscarriages of justice led to the establishment of theCriminal Case Review Commissions in England and Wales and Scotland. See L. Elks, Righting Miscar-riages of Justice? Ten Years of the Criminal Cases Review Commission (Justice: London, 2008) ch. 4; P. Duff,‘Straddling Two Worlds: Reflections of a Retired Criminal Cases Review Commissioner’ (2009) 72MLR 693. In the United States, various Innocence Projects seem to have been highly influential, seeB. Garrett, Convicting the Innocent: Where Criminal Prosecutions Go Wrong (Harvard University Press:Cambridge, MA, 2011).

20 Edmond and Roberts, above n. 6.21 These tendencies—spurred in part by crime control and law and order campaigns—may have

surreptitiously diminished the strength of proof ‘beyond reasonable doubt’.

evidence of previous convictions, similar fact and hearsay—or prevented certaininferences—such as about propensities where evidence was limited to credibility,about silence in the face of accusation, or the presentation of alibis late in theprocess of prosecution—have been wound back or qualified to facilitate admission(for previously impermissible purposes).22 The admission of incriminating expertopinions is merely part of this more inclusive ethos.23

These principles, goals and trends represent a range of legally orthodoxcommitments that feature routinely in key judgments and the extra-curialpronouncements of senior judges, politicians and legal scholars.24 They shouldnot, however, be understood as some timeless and uncontested values or prescrip-tions.25 There are, significantly, dissenting voices. Populist critiques of criminaljustice, along with victims’ groups, often rail against the rights and protectionsafforded to criminal defendants.26

Against this backdrop this article now turns to consider the admissibility offorensic science and medicine evidence. In doing so it is my intention to takeseriously the professed commitment to factual rectitude and the obligations onthe state in circumstances where there are persistent and unsettling questionsabout the trial and the jury. This article is, therefore, a response to the inefficient,inconsistent and ineffective ways in which incriminating expert opinion evidence



22 ‘Free proof’ captures an approach to decision-making (or fact-finding) that is naturalistic in thesense that it is devoid of restrictive rules and rational in the sense that it is only attentive tologically relevant evidence, see Twining, Rethinking Evidence, above n. 13 at 195. Examples of theexpansion of admission and use can be seen in the English response to a suspect’s silence (CriminalJustice and Public Order Act 1994, ss. 34–36) and in Australian responses to similar fact andpropensity evidence (Uniform Evidence Law, ss. 97, 98).

23 Liberal admissibility practices, it should be noted, produce guilty pleas, and generate massive costsavings at the front end of criminal justice processes.

24 See e.g. H. L. Ho, A Philosophy of Evidence Law: Justice in the Search for Truth (Oxford University Press:Oxford, 2008); A. Stein, Foundations of Evidence Law (Oxford University Press: Oxford, 2005).

25 See J. Langbein, The Origins of the Adversary Trial (Oxford University Press: New York, 2003) for adiscussion of the Anglo-American tradition and C. Allen, The Law of Evidence in Victorian England(Cambridge University Press: Cambridge, 1997) for a critical response to Bentham’s allegedinfluence on English evidence and procedure.

26 Two centuries ago Jeremy Bentham asserted that ‘[e]vidence is the basis of justice: to excludeevidence is to exclude justice’ (J. Bentham, The Works of Jeremy Bentham (ed. J. Bowring), 11 vols.(Edinburgh, 1843) VII 24). Larry Laudan, something of a neo-Benthamite, has recently suggestedthat many exclusionary rules, such as those regulating reliance on similar fact evidence, hearsay,and possibly rules pertaining to the admission of opinion evidence, effectively give those accusedof crimes too much protection when used in conjunction with the demanding standard of proof(i.e. beyond a reasonable doubt). See L. Laudan, Truth, Error, and Criminal Law: An Essay in Legal Episte-mology (Cambridge University Press: Cambridge, 2005). Laudan’s philosophical orientation islargely indifferent to socio-legal and empirical research. Compare Part 1 and Edmond and Roberts,above n. 6.

is produced, adduced, examined and assessed in modern criminal trials andappeals.

3. Admissibility standards (in the major common law jurisdictions)

Notwithstanding the primacy of ‘truth’ in all jurisdictions, in England and Wales,Australia and New Zealand the concept of reliability has not played an importantrole in admissibility jurisprudence or practice pertaining to expert opinionevidence.27 While the ‘reliability’ of expert opinion is sometimes mentioned injudgments, especially in the aftermath of (notorious) miscarriages of justice, theLaw Commission of England and Wales recently summarised ‘its practical effect[as] largely illusory’.28 Appellate courts in England and Australia have expresslyrejected reliability, especially any kind of strict requirement linked to specificcriteria.29 Interpreting s. 79 of the Evidence Act 1995, loosely based on the originalr. 702 of the Federal Rules of Evidence (US), the New South Wales Court of CriminalAppeal explained that the ‘focus of attention must be on the words “specialisedknowledge”, not on the introduction of an extraneous idea such as “reliability”’.30

The Law Commission of England and Wales has, however, recently advocated theintroduction of admissibility criteria similar to those associated with Daubert vMerrell Dow Pharmaceuticals Inc. (see Part 1, Section 1). The Law Commissionproposes a revised admissibility regime, with reliability transformed into athreshold issue for admissibility. In addition to codification of the prevailingcommon law rules, the draft of the Criminal Evidence (Experts) Bill states that‘expert opinion evidence is admissible in criminal proceedings only if it is suffi-ciently reliable to be admitted’ (cl. 2: italics added). The meaning of ‘sufficientlyreliable’ is elaborated in cl. 4 and Part 1 of the Schedule, though it is largelyconsistent with the kinds of indicia associated with Daubert and the revision of theFederal Rules of Evidence in 2000. The Report also recommends that ‘in excep-



27 Trial and appellate judges maintain confidence in the trial, trial safeguards, lay jurors andappellate courts to expose and overcome problems with expert opinion evidence.

28 Law Commission of England and Wales, Expert Evidence in Criminal Proceedings in England and Wales,Law Com. Report No. 325 (2011) para. 3.3 (hereafter ‘Report’). See also Law Commission of Englandand Wales, The Admissibility of Expert Evidence in Criminal Proceedings in England and Wales: A NewApproach to the Determination of Evidentiary Reliability, Law Com. Consultation Paper No. 190 (2009)(hereafter ‘Consultation Paper’). See Part 1, Sections 1(a) and 2(d).

29 R v Robb (1991) 93 Cr App R 161; R v Reed [2009] EWCA Crim 2698; R v Henderson [2010] EWCA Crim1269; R v Weller [2010] EWCA Crim 1085; R v Luttrell [2004] EWCA Crim 1344.

30 R v Tang (2006) 65 NSWLR 681 at [137]. Compare the Supreme Court of the United States in KumhoTire Co. v Carmichael, 526 US 137 at 147 (1999): ‘In Daubert, the Court specified that it is the Rule’sword “knowledge,” not the words (like “scientific”) that modify that word, that “establishes astandard of evidentiary reliability”.’

tional circumstances’, court-appointed experts might be used to assist withadmissibility determinations (cl. 9; and see Section 6 below).

A major problem, as experience with reliability standards in the United States andCanada makes clear (see Part 1, Section 2(b)), is that common law judges have beenreluctant to apply them to exclude incriminating expert opinion evidenceadduced by the state. In response, this article proposes a mechanism to assist withadmissibility decision-making. It aims to help relieve some of the mostproblematic issues associated with admissibility, and thereby practically assist theparties and courts in selecting, presenting and contesting expert opinionevidence—particularly techniques and opinions that are unreliable or of unknownreliability.

Reiterating the discussion in Part 1, admissibility decision-making is importantbecause of the frailty of the trial safeguards, serious and unanswered questionsabout the tribunal of fact and recent and unsettling criticism of many areas offorensic science and medicine—particularly the pattern recognition andcomparison ‘sciences’.31

4. A multidisciplinary advisory panel and its virtues

In response to problems with insufficiently reliable techniques and opinions I amproposing that admissibility practice might be enhanced through the assistance of amultidisciplinary advisory panel (or MAP).32 Some of the difficulties with the morecontroversial forms of forensic science and medical evidence might be reducedthrough recourse to an advisory mechanism that is independent of the parties,their experts, and the particular litigation.33 I want to emphasise that thisproposal is developed in response to formal admissibility standards, espousedlegal principle and system aspirations—both the goal of a fair trial and an accurateverdict—rather than some reductive model of (authentic) science or recourse tosimplistic models of public technical deficits.



31 National Research Council (NAS), Strengthening the Forensic Sciences in the United States: The PathForward (National Academies Press: Washington DC, 2009). See also G. Edmond, ‘Actual Innocents?Legal Limitations and their Implications for Forensic Science and Medicine’ (2011) 43 AustralianJournal of Forensic Sciences 177.

32 Ideally, the MAP would operate within a statutory framework that facilitates legal recognition ofany advice (regardless of whether it is accepted) as well as provide funding. This could include amore accommodating form of judicial notice, see E. Cheng, `Independent Judicial Research in theDaubert Age’ (2007) 56 Duke Law Journal 1263.

33 More generally, see S. Jasanoff, The Fifth Branch (Harvard University Press: Cambridge, MA, 1990); R.Pielke, The Honest Broker (Cambridge University Press: Cambridge, 2007).

I envisage a MAP composed of several experts from a variety of established fields,such as chemistry, the biosciences, epidemiology or medicine, engineering,statistics or mathematics, experimental psychology, along with the forensicsciences, forensic pathology (i.e. practising forensic scientists and pathologists),legal practice or the judiciary, and an academic lawyer—perhaps 8 to 12 in total.Similarities with the NAS committee should be obvious.34 The precise compositionof the panel is less important than membership being dominated by highlyqualified (non-forensic) scientists of demonstrated ability. Selection isimportant.35 Because of the ease of partiality allegations and potential contro-versy, most of the scientific and biomedical researchers, engineers andstatisticians should be independent of the institutional forensic sciences and haveno conflicts of interest.36 While the panel should be empowered to appointspecialist members—on an ad hoc basis to assist with particular references orissues—to the extent that the panel struggles with specialised knowledges, thepossibility of serious jury comprehension or engagement through the course of atrial would seem to be, at best, fraught. Not including the most specialisedindividuals—as standing members of the panel—should not prevent the generalistpanel from fulfilling its primary responsibility: to provide practical, accessibleand timely advice to assist courts with admissibility determinations.37

The MAP would be exclusively in the business of producing written advice based onnarrow references.38 A reference would identify an issue or area that was perceivedas troublesome; such as the use of CCTV and surveillance images, or foot and shoemarks, or the value of cross-lingual voice comparisons from surveillancerecordings, for the purpose of identification. Precisely how any reference would beinstigated is an open question, though ideally should be linked to emergingtechniques and their applications rather than in response to individual trials (andsome particular factual nexus). The panel could receive references fromAttorneys-General, senior judges, prosecutors or defenders, forensic science insti-tutions (or some alternatives) and/or have discretion to undertake themunilaterally.39



34 National Research Council (NAS), above n. 31 at v.35 There are many advisory panel and committee selection models to choose from. See e.g. W. Bijker,

R. Bal and R. Hendriks, The Paradox of Scientific Authority (MIT Press: Cambridge, MA, 2009).36 This does not mean that they should be unfamiliar with the issues, or controversies, in the forensic

sciences.37 However, there should be scope to add one or more specialist panel members to assist with specific

references.38 I have used italics (e.g. advice) where referring to the formal published advice produced in response

to the reference by the panel.39 The latter method seems preferable, as it would allow the panel to manage its own abilities and

resources.

Like the NAS committee, members of the panel would be selected though notremunerated. This would limit the cost of maintaining a panel of highly qualifiedindividuals, but simultaneously restrict the number of references to around oneto three per year. Ideally, a reference should be completed in around four to sixmonths. Realistically, the advice would be drafted by (or under the supervision of) acouple of panel members responsible for chairing each reference. The panel wouldconvene to discuss the reference, the relevant research literatures, negotiate itsdecision(s) and edit the advice.40

References would involve a general review of a technique (or related techniques)already in use or whose use in criminal proceedings (in the plural) is anticipated.The confined nature of a reference would seem to render the panel and itsinquiries best suited to emerging techniques and simmering controversies. Areference would lead to the production of a short written advice. The reference andadvice would be constrained by published research (and research made available),the generalist composition of the panel, as well as temporal and resourceconstraints.41 In effect, the panel would undertake a review, including a wide-ranging literature review and assessment of techniques, sensitive to experimentaland methodological norms. The purpose is to facilitate a reasonably quickassessment based on what is empirically known about techniques and interpreta-tions.42 Members of the panel would not, therefore, undertake any experiments ortesting. Rather, they would aim to produce a consensus statement focused on thereliability (and limits) of a technique and derivative opinion.43



40 It would be possible for a draft of any advice to be reviewed by a separate group of scientists (throughsome kind of independent peer review process), and there could even be provision for publiccomment on the draft—with an expectation that the panel would respond formally to criticisms andcomments. These possibilities might enhance the social and epistemic legitimacy of the process andthe resulting advice, but will simultaneously add delay, cost and complication to its production.Ultimately, there will be a need to balance the provision of expedient advice to assist with admissi-bility decision-making against the willingness of eminent scientists (and others) to participate in aprocess that produces conclusions conceived as sufficiently robust and socially legitimate.

41 The Intergovernmental Panel on Climate Change affords an example of a formalised approach tothe review of literature (and data). The Cochrane Collaboration offers another example wheredifferent types of medical research (e.g. meta-analyses, randomised clinical trials, case-controlstudies, clinical opinion and other types of studies) are placed in a formal hierarchy—with ‘expertopinion’ on the lowest rung.

42 Focusing on a technique or type of evidence in isolation prevents the many different types ofinformation (including admissible and inadmissible evidence) available during trial andsentencing from contaminating the assessment of reliability.

43 Consensus is not strictly necessary and there should be scope for dissent or disagreement. It ispossible that the panel consensus may diverge from any ‘professional’ consensus in the literature.Although this would seem to be highly unlikely except where supporting literatures were notempirically based. The panel would not be favourably disposed to previous admissibility decisionsas support for reliability.

In the end, it would be useful for the panel to conclude that a technique is either:

reliable (or basically reliable)

OR

unreliable (or of unknown reliability)44

The focus on the reliable/unreliable dichotomy, and requiring a categorical call, isindexed to legal admissibility needs.45 The advice is intended to provide practical,timely and authoritative information of a general nature rather than exhaustivespecialist assessment or an attempt to represent the interests of ‘stakeholders’.46

The advice following a reference must be succinct and serviceable. The panelshould endeavour to produce a very short document (say, a maximum of eightpages) that reviews and evaluates relevant research literatures. The advice shouldoffer an assessment of the reliability—in the sense of epistemic value—of thetechnique(s) and derivative opinions. Should a technique be classified as unreliableor of unknown reliability, the advice could include a description of the types ofstudies or refinements that the panel envisages might be required to sustain (orenhance) reliability. An advice should not discuss legal issues, such as admissi-bility, that remain the exclusive province of the courts. The advice should bewritten for a general, rather than technical, audience and should be madepublicly available (as a free PDF). Given the terms and nature of the reference, theadvice would be technical, but not a purely scientific document. The terms andconditions of production would make the advice a peculiarly law–scienceknowledge product.47 It would be a knowledge product engaging with relevantliteratures, research and specialist communities and summarising findings in a



44 The categories ‘unreliable’ and ‘of unknown reliability’ are quite different, though both wouldseem to be insufficiently reliable and raise serious difficulties for the accusatorial trial as well asreal risks for the accused.

45 Complications may be created by requiring the panel to offer its written advice in terms of jurisdic-tional admissibility standards—that is, by making a call on reliability. There is, however, anadvantage in requiring the courts and those beyond the courts to adopt a shared language ratherthan creating an artificial attempt to avoid shared terminology. In addition, and importantly, thecourts are not, in the end, bound to adopt the advice. The MAP has no formal powers in relation toadmissibility. As Section 8(a) below explains, there is scope for reasoned derogation with theexpectation that failure to follow any advice will generate compelling, publicly accessible reasons.

46 Unlike the advisory panel of the Forensic Science Regulator (UK), the MAP would not incorporatestakeholder participants, although a few respected forensic science and legal practitioners wouldbe included among the panel members. Stakeholders would be entitled to make submissions and,depending on the organisation of the references and review processes, possibly able to commenton any draft advice.

47 See, e.g., S. Jasanoff (ed.), States of Knowledge (Routledge: London, 2006); Bijker et al., above n. 35.

manner that is simultaneously accessible to non-specialist audiences. Throughdissemination of the advice the panel would be publicly accountable.

The proposal trades on provision of an indicative assessment of the value of atechnique (or techniques). Interested groups—that is, those with relevanttechnical, methodological and statistical abilities—such as the institutionalisedforensic sciences, their professional bodies, and academic commentators, shouldbe able to make submissions to a reference—with a similar status to amicus curiaebriefs—and the panel should have the ability to hold brief (public) hearings, whereinformation is solicited following invitation (or upon request to make asubmission).48 However, such hearings should be held at the panel’s discretion.The panel should have no formal powers to call witnesses or acquire materials.Because the review would be primarily based on publicly available research, andbecause there is increasing sensitivity to reliability standards within forensicscience communities (following the NAS report), the time available to makesubmissions, once a reference is advertised, should be short—in the order of aboutfour to eight weeks.49 The failure to make a submission or provide evidence insupport of a technique when requested might, in the absence of publishedempirical information, contribute to an adverse finding.50

In many cases all that a panel will be able to report is that the kinds of studies thatare necessary to ground a technique or interpretation have not been undertaken,are inadequate, and/or that existing practice is methodologically suspect.51 In theabsence of credible research or evidence to support a technique or approach, it isnot the responsibility of the panel to stand in, or vouch, for missing research or tospeculate about the probative value of unsupported techniques and derivative



48 See, e.g., US Supreme Court Rules, r. 37, governing amicus curiae briefs in the United States.49 Materials in the private domain, including techniques that are relevant to national security or

protected under intellectual property and trade secrecy arrangements, may be withheld from theMAP. There would seem to be few reasons not to provide security clearance to members of thepanel even if it meant the resulting advice might be redacted.

50 Much of this information should already be assembled and incorporated into expert reports tomake the support for conclusions more transparent. Indeed, many jurisdictions require suchinformation, e.g. Practice Note CM 7—Expert Witnesses in Proceedings in the Federal Court of Australia,2011, available at <www.fedcourt.gov.au/how/prac_direction.html>, accessed 26 April 2012. Seealso B. Found and G. Edmond, ‘Reporting on the Comparison and Interpretation of PatternEvidence: Recommendations for Forensic Specialists’ (2012) 44 Australian Journal of ForensicSciences 1.

51 The panel might be able to develop a grading of studies, based around the evidence-based medicine(EBM) rankings, to assist those reviewing or undertaking studies to appreciate the relative value ofdifferent kinds of approaches to validation and reliability studies. See e.g. D. Atkins et al., ‘GradingQuality of Evidence and Strength of Recommendations’ (2004) 328 British Medical Journal 1490.

opinion (even from those represented as ‘experienced’).52 It is unlikely that a panelwill resolve technical controversy.53 Nevertheless, the panel and the interest inempirical evidence should help to prevent new techniques and approaches fromentering courtrooms prematurely—before they are demonstrably reliable. A MAPwill offer socially credible, independent perspectives on the value of forensicscience and medical techniques and interpretations through reference to researchliteratures and studies. This is precisely what courts require, but seldom receivefrom the parties or their experts, when obliged to make admissibility determina-tions during criminal proceedings.

Notwithstanding the provision of this information, a MAP is generally comple-mentary to common law institutions and practice. It does not radically disrupt theparty-oriented nature of the accusatorial trial and pre-trial investigation andpreparation. Rather, it provides a resource—for all of the parties, as well as the trialjudge, investigators and forensic scientists—usually removed from any particularinvestigation and prosecution.54 A MAP does not usurp the role of judges or theconstitutional role of the tribunal of fact (whether judge or jury). The map (andadvice) leaves admissibility determinations (i.e. gatekeeping) to trial and appellatejudges and the assessment of admissible expert opinion evidence to jurors andappellate courts. Advice, ordinarily, will be more authoritative than the opinion ofa party expert (or court-appointed expert). It is far more likely to have ramifica-tions beyond the individual case (or issue) and therefore has the potential to helpstandardise decisions as well as enhance institutional efficiencies.55 Moreover, itmight influence investigative practice and improve the quality of expert opinionevidence in plea negotiations as well as pre-trial negotiations and trial planning. AMAP, in effect, is likely to exert influence in a preventative capacity.56



52 I do not want to suggest that experience is not valuable, but rather to challenge the assertion thatexperience alone is sufficient to ground admissibility for techniques. There is, instead, a need forempirical assessment. That is, we require demonstrably reliable techniques used by experiencedand proficient individuals.

53 Contrast claims made about the science court and other proposals and experiments. See, e.g., B.Casper and P. Wellstone, ‘Science Court on Trial in Minnesota’ in B. Barnes and D. Edge (eds.),Science in Context: Readings in the Sociology of Science (Open University Press: Milton Keynes, 1982); J.Martin, ‘The Proposed “Science Court”’ (1977) 75 Michigan Law Review 1058; Note, ‘Confronting theNew Challenges of Scientific Evidence’ (1995) 108 Harv L Rev 1481.

54 Compare the use of court-appointed panels in court in L. Hooper, J. Cecil and T. Willging, ‘AssessingCausation in Breast Implant Litigation: The Role of Science Panels’ (2001) 64 Law & ContemporaryProblems 139.

55 Court-appointed experts and concurrent evidence, in contrast, tend to feature relatively late inproceedings.

56 This is a direct analogy with preventative medicine and the social and system advantages accruingfrom early interventions (rather than corrections).

A MAP will help to develop greater consistency across legal responses to a range oftechnical and scientific fields and techniques.57 A MAP can help with consistencybecause many issues, such as the way to minimise contextual effects, andinterpret and express results, reoccur—especially in comparison and patternmatching domains. In consequence, even a small number of references may exerta disciplining influence on the provision of forensic science and medical evidenceand legal practice. The advice following a critical reference (i.e. unreliable),especially if it leads to the exclusion of evidence, might also stimulate research(and the provision of resources for research).58

A MAP will help to reduce the significance of resource disparities between partiesand their accentuation through the trial and appeal. It reduces the degree towhich an accused person is dependent upon the (assumed) efficacy of trialsafeguards and jury comprehension in the face of complex and technical evidencewhere limitations are not necessarily canvassed. In some proportion of cases theimpecunious accused will no longer be required: to persuade a lay tribunal of factthat techniques in wide use and routinely admitted by courts may be unreliable;to explain cognitive biases and the difficulty of overcoming them; or to explainthe limits of complacent investigative experience and confident opinions—allduring the course of an accusatorial proceeding. For the advice will providecriminal justice personnel and the parties (particularly the defence) with access toa range of relevant scientific and technical literatures that might not otherwisemake their way before the trial judge (or court of appeal) or tribunal of fact. Advicefrom a MAP reduces the court’s dependence on the parties and their experts forbalanced information about techniques and the expression of opinions.

A MAP also transcends the constraints and interests of particular fields and disci-plines. It is not dependent upon parochial professions (for example, tool markexaminers or arson investigators).59 Rather, a MAP makes such groups and theirtechniques accountable to a highly qualified and technically sophisticatedassembly of experts.60



57 The MAP is technique-oriented rather than case-based, and so offers the potential to improve theconsistency in response to particular techniques, as well as the forensic sciences more generally.

58 See, e.g., B. Silverman, Research and Development in Forensic Science: A Review (London: Home Office,2011) paras. 4.4–4.5.

59 The panel should be willing to hear from and take submissions from such groups, but should beled by the results of published empirical studies.

60 Court-appointed experts, in contrast, are likely to be selected from the field—even where itsexistence and/or abilities are contentious. Who, for example, should be selected as a court-appointed expert in relation to contested ‘facial mapping’ evidence or an alleged baby-shakingdeath? See also Section 6 below.

The MAP provides a forum and a level of influence that is conducive to theinvolvement of eminent experts. The panel is not adversarial and members are notsubject to credibility challenges via cross-examination. This may encourage highlyskilled and capable experts, including some who might not wish to appear in anadversarial setting—even as court-appointed experts—to participate as part of theircivic (or professional) service.

It is not suggested that the panel, given its multidisciplinary composition,constrained time frame and resource limitations, is well suited to the exhaustivereview of very widely used techniques such as DNA profiling or the analysis oflatent fingerprints. The importance of these techniques along with theircontinuing and central role in a range of investigative and surveillance activities(for example, border control and criminal databases) warrants more expansivesupervision and review by better-resourced and more specialised (as well asnon-specialist) audiences. While a panel might offer useful advice on the intro-duction of techniques in their early stages, as techniques become mainstream andinstitutionalised there will often be a need for more extensive (and ongoing)assessments and negotiations around standardisation of practices, applications,interpretations and reporting. These, as NAS reports on DNA evidence suggest, canbe expensive, incredibly complex, and controversial.61

Rather than black-boxing technical assessment, the MAP’s conspicuous limita-tions expose many of the challenges involved. The production of the publiclyavailable advice represents a reasonably transparent review that will be suffi-ciently robust for admissibility purposes though without pretensions of beingcomprehensive or definitive. The potential for refining advice and the possibility ofcontravention (in admissibility hearings or during trials) should be seen aspotential virtues—both epistemological and institutional.62 Moreover, the MAPdoes not require the transformation of lawyers and judges into scientists oramateur sociologists and philosophers of science—desirable as such transforma-tions might appear.63 Instead, the MAP is focused on practical legal needs, namelyproviding information about the reliability of techniques. It is not primarilyoriented to the needs of forensic scientists or investigators, or the exigencies of



61 See the two National Research Council (of the NAS) reports on DNA—DNA Technology in ForensicScience (1992) and The Evaluation of Forensic DNA Evidence (1996)—and the accounts by J. Aronson,Genetic Witness: Science, Law, and Controversy in the Making of DNA Profiling (Rutgers University Press:Piscataway, NJ, 2007); D. Kaye, The Double Helix and the Law of Evidence (Harvard University Press:Harvard, 2010).

62 Jasanoff, above n. 9; Jasanoff, above n. 47.63 Consider the dissent by Rehnquist CJ in Daubert (Daubert v Merrell Dow Pharmaceuticals Inc., 509 US

579 at 598–601 (1993)). See also S. Haack, ‘An Epistemologist in the Bramble-Bush: At the SupremeCourt with Mr. Joiner’ (2001) 26 J Health Politics, Policy & L 217.

particular prosecutions. Nevertheless, any advice might help to prevent investi-gators, technicians and scientists from entering criminal courts and relying upontheir qualifications, institutional authority or untutored experience.

The intervention of the MAP might stimulate legal interest in reliability and helpto transform debates around scientific knowledge from credibility challenges tocontests around experimentation and published research, accuracy and errorrates, as well as the proficiency of individual scientists and the culture of investi-gative institutions. It might also: temper confidence in experience and the overallstrength of particular cases (see Part 1, Section 2(d)(ii)); deflect attention fromissues such as whether the defence has access to an expert (i.e. a rebuttal expert);operate independently of trial safeguards; help to improve the quality of evidenceincluded in accusations that lead to guilty pleas (where techniques and opinions,however reliable, are rarely subjected to scrutiny); and is largely indifferent topersistent questions about lay abilities with technical, complex and statisticalevidence (see Section 7 below). The existence of independent and socially credibleadvice might encourage prosecutors, in their capacity as ‘ministers of justice’ (andmodel litigants), to refrain from adducing incriminating opinions derived fromtechniques that are not demonstrably reliable.64

Advice from the MAP is sufficiently reliable for admissibility purposes. Courtscould hardly be criticised for deferring to the advice of a MAP upon consideredreview. The advice facilitates, and perhaps encourages, deference, but simulta-neously enables principled derogation (see Section 8(a) below).65 Courts are notbound by the advice but would be expected to explain departures from publishedadvice and do so at risk to their own socio-epistemic legitimacy. Such reasonswould help to make legal decision-making more transparent, publicly account-able and consistent with what is known beyond the courts.66 Independent advicemay make it easier for courts to take politically difficult and institutionally unset-tling decisions—such as excluding forensic science techniques that have featuredprominently in investigations and prosecutions under more liberal admissibility



64 Boucher v The Queen (1954) 110 CCC 263. See also M. Saks, ‘Scientific Evidence and the EthicalObligations of Attorneys’ (2001) 49 Clev St L Rev 421; M. Raeder, ‘See No Evil’ (2007) 76 Fordham LRev 1413.

65 Allen and Miller, above n. 10.66 The advice provides an opportunity for relevant non-legal (i.e. exogenous) expertise to influence

criminal justice practice (especially admissibility) in a manner that produces greater consistencybetween what is known outside the courts and what is presented inside the courts—so-called equili-bration. See G. Edmond, ‘Bacon’s Chickens? Re-Thinking Law and Science (and IncriminatingExpert Opinion Evidence) in Response to Empirical Evidence and Legal Principle’ in J. Gleeson andR. Higgins (eds.), Constituting Law: Legal Argument and Social Values (Federation Press: Sydney, 2011).

regimes.67 English courts, for example, may need prompting to exclude ‘facial andbody mapping’ evidence (i.e. the use of images for the purposes of ‘identification’)in the absence of empirical support confirming ability and accuracy.68

5. Potential vulnerabilities, institutional resistance and procedural obstacles

There may, of course, be difficulties with a MAP. The pragmatic, ‘rough and ready’,advice will be vulnerable on several levels. Potential complications include:

• Setting the terms of reference for the panel, in relation to anyparticular reference, may prove challenging.

• It may not be possible to achieve broad consensus in relation to manytechniques (and their applications).

• The relevant literatures may be difficult to identify, let alone bringtogether in order to synthesise, summarise or draw conclusions from.Here, the default position would seem to be that if the panel is in doubtabout whether empirical studies or research literatures support thevalue of a technique they should cast doubt on reliability (e.g. ‘ofunknown reliability’).

• It is not insignificant that reliability often possesses different meaningsfor lawyers and among scientists. The meaning of ‘reliability’, as asynonym for trustworthiness or the quality of forensic scienceevidence, may make it difficult to produce a categorical call; especiallywhere a technique can be used in a variety of ways and the expressionof results might range from apparently very probative to not veryprobative at all.69



67 The advice may also help to temper, or productively channel, increasingly robust expressions ofconcern from scientists, social scientists, statisticians and academic commentators (individuallyand collectively), in response to the perceived irrationality (and unprincipled nature) of some areasof contemporary legal practice. See I. Evett et al., ‘Expressing Evaluative Opinions: A PositionStatement’ (2011) 51 Science & Justice 1 and attempts at rapprochement such as M. Redmayne, P.Roberts, C. Aitken and G. Jackson, ‘Forensic Science Evidence in Question’ [2011] Crim LR 347; M.Saks and D. Faigman, ‘Failed Forensics: How Forensic Science Lost its Way and How it Might YetFind it’ (2008) 4 Annual Reviews of Law & Social Science 149; M. Saks and J. Koehler, ‘The ComingParadigm Shift in Forensic Identification Science’ (2005) 309 Science 892; G. Morrison, ‘TheLikelihood-Ratio Framework and Forensic Evidence in Court: A Response to R v T’ (2012) 16 E&P 1.

68 See Atkins v The Queen [2009] EWCA Crim 1876, [2010] 1 Cr App R 8; R v Weighman [2011] EWCA Crim2826; Note, ‘Atkins v The Emperor: The “Cautious” Use of Unreliable “Expert” Evidence’ (2010) 14 E&P146.

69 Compare the very different responses to footprints and images in R v T [2010] EWCA Crim 2439,[2011] 1 Cr App R 9 and Atkins v The Queen [2009] EWCA Crim 1876, [2010] 1 Cr App R 8.

• There is the problem of what to do with similar (or analogous)techniques not specifically covered by the reference and advice. Where,for example, a reference assessed the use of CCTV and surveillanceimages for identification purposes (around face or body shape), couldthe advice be usefully extended (or applied) to sub-features of identityderived from images, such as gait, height, race, gender or clothing andmannerisms (for example, handedness or smoking)?

• Regardless of the advice, once one court admits some technique oropinion, even upon a limited basis, it will be difficult for common lawjudges not to use that decision as authority for general admission andother purposes (as in the CCTV example, above). Such precedent mayfacilitate admissibility creep.70

• As techniques are refined or new research is undertaken the panel mayneed to revisit past references and modify earlier advice. Inconsistenciesbetween any advice and scientific studies, even if emerging over time,may be used to challenge the panel and its performance; allowing thepanel to be cast as inconsistent, incompetent or unrepresentative.71

• Panel certification (i.e. a ‘reliable’ finding) may bolster (or reify) sometechniques and forms of expertise thereby making judges and juriesless sympathetic to criticism however well conceived.

In addition, the composition and selection of the panel may be controversial (or,perhaps more accurately, politicised). Those who are specialists in a particulardomain may challenge the findings of the more generalist panel. Such disagree-ments, of themselves, may be informative in relation to the admission of opinionevidence to be presented before a lay tribunal of fact. If, for example, the panelcomes to a different conclusion to, or is unpersuaded by, specialists, this mightinform admissibility decision-making and the exercise of judicial discretionsaround probative value, the ability to confuse or mislead (i.e. unfair prejudice) andwhether such evidence will waste time and resources. The panel, it should beremembered, will be encouraged to make categorical calls (i.e. ‘unreliable’ asopposed to ‘reliable’) in order to assist with the mundane practice of admissibilitydecision-making. Where the panel cannot achieve consensus the advice and itsimplications might not be particularly constraining on admissibility practice butwill still provide a useful resource for parties, trial and appellate judges.



70 This was the experience with the initially qualified admission of bite-mark evidence in the UnitedStates, see D. M. Risinger, ‘Navigating Expert Reliability: Are Criminal Standards of Certainty BeingLeft in the Dock?’ (2000) Alb L Rev 99; E. Beecher-Monas, ‘Reality Bites: The Illusion of Science inBite-mark Evidence’ (2008) 30 Cardozo L Rev 1369.

71 Research stimulated by the publication of the panel’s advice could hardly be considered adisadvantage, even if it generates controversy or complicates the utility of the advice.

There will be a need to insulate the panel and its decision-making processes fromscrutiny. The aim is to achieve criminal justice objectives; particularly avoidingthe risk of error, through the provision of publicly accessible informed assistance,and a reduction in the volume of gratuitous and repetitive litigation. Actualadmissibility contests should focus on the evidence supporting techniques andabilities rather than generate subsidiary issues around alleged procedural irregu-larities or conflicts of interests associated with the panel, its members andprocesses.72 The non-determinative nature of the advice means that there is noneed for evidence of the panel’s decision-making processes to be open to review inpreliminary hearings, trials and appeals. Similarly, members should not becompellable.73 Rather, the advice itself should form part of, and inform, any contestover admissibility.

Whether key institutions and organisations—such as the police, state laboratories,private service providers and manufacturers of equipment—will cooperate withthe panel is also uncertain, especially in the absence of powers of compulsion. Theinstitutionalised forensic sciences might attempt to circumvent the panel by notparticipating in references in order to maintain the historically accommodatingadmissibility hearing (and trial) as the preferred site for the legal assessment offorensic science and medicine. Forensic scientists will tend to fare betterin criminal courts because: trial and appellate judges are generally favourablydisposed to the state’s institutions and personnel; trial and appellate judges oftenhave access to inadmissible information about the accused; those reviewing theexpert opinion evidence are not technically trained; and the resources available tothose opposing admission—frequently publicly funded defence lawyers withouttechnical abilities or expert assistance—are not merely uneven, but often negli-gible.74

Institutional resistance may also be influenced by the manner in which judges areselected. Criminal judges are drawn largely from the ranks of prosecutors (andsuccessful civil/commercial lawyers). Professional, political and institutionalideologies and pressures, along with long socialisation and various media influ-ences, will continue to shape criminal justice practices. It is not clear whether anindependent MAP will help to make judges either less trusting of the state or moretolerant of expert opinion evidence adduced by criminal defendants—especially



72 This is not to suggest that processes should not be routinised and fair.73 S. Hilgartner, Science on Stage (Stanford University Press: Stanford, CA, 2000) and Bijker et al., above

n. 35.74 See L. Baum, Judges and Their Audiences (Princeton University Press: Princeton, NJ, 2006).

methodological criticism. Judges might develop a range of non-epistemic meansof managing external interference.75

The cost of operating a MAP is an important issue and potential justification forinaction. While a panel of experts might appear costly, there is no reason to thinkthat it would be particularly expensive, especially when weighed against potentialsystem benefits (and alternatives such as court-appointed experts). Assumingthat membership of the panel was without remuneration, apart from expensesassociated with occasional meetings, the main costs would be for secretarial assis-tance, information technology, research and drafting support. Any ‘additional’expense associated with a panel should be contrasted with the system costscreated by the continuing adduction, primarily by the state, of expert opinionsthat are not demonstrably reliable. Unreliable and speculative forms of expertopinion represent a considerable financial imposition that is compounded by thecost of trial preparation, the public funding of rebuttal experts (or advisers), timespent preparing and cross-examining, time devoted to jury instructions, andappeals on what are very often reliability-related issues. Taking ‘facial mapping’ inEngland as an example, the advice of a MAP may have saved millions of pounds inexpert fees and court time over the previous decade. Similarly, early literaturereviews and advice on shaken babies or the implications of multiple infantdeaths in one family might have prevented several notorious convictions thatwere eventually reversed on appeal.76 In these later cases, just the compensationpayments ran into millions of pounds. Authoritative advice from a panel mayprevent a wide range of expenses being incurred by investigators, parties and thecourts and, more importantly, reduce the likelihood of convicting the innocent (orsubstantially unfair criminal prosecutions) and the resulting institutionaldisruption.

The question of how the court identifies and considers (or accommodates) formaladvice also requires consideration. This might be statutory or one party mightchoose to adduce and rely upon the advice as part of a proffer of evidence orreliability challenge. The advice would presumably be available at any admissi-bility hearing.77 Whether the advice should be available to the jury would need tobe determined on a case-by-case basis. The fact that an advice will often advantageone side in a dispute over admissibility is entirely proper. Ideally, the prosecutor



75 See Part 1, Section 2(d).76 See, e.g., R v Clark [2003] EWCA Crim 1020; R v Cannings [2004] EWCA Crim 1. More generally, see E.

Cunliffe, Murder, Medicine and Motherhood (Hart: Oxford, 2011).77 If a MAP were established by statute, relevant advice should be formally admissible during admissi-

bility hearings, at the very least.

should bring any relevant information, whether pro or contra, to the attention ofthe court and the other parties.

It is not possible to accommodate the potential advantages and disadvantageswithin some simple matrix or balancing exercise. The extent to which potentialdifficulties materialise will depend in large part on the members of the panel,the cooperation of the institutionalised forensic sciences and increasinglycommercial (i.e. private) providers, and the responses of trial and appellate judgesand lawyers.78

6. The limits of impartiality (and court-appointed experts) and narrow

consensus

Before moving to the conclusion, I want to address briefly two issues that infectthe Law Commission’s discussion and proposals for reform. Initially, I want tosuggest that the idea of impartiality, and recourse to impartial court-appointedexperts, are of less value than a panel of experts focused on evidence of reliability.Thereafter, I want to raise the persistent and potentially disruptive issue of thepublic understanding of expert opinion evidence.

In response to acknowledged problems with the admission of too much unreliableexpert opinion evidence,79 the Law Commission Report emphasised the value ofimpartiality and proposed using court-appointed experts to assist with admissi-bility determinations.

When considering the reliability of very complex expert opinionevidence under our reliability test, a judge might in exceptional casesrequire the assistance of an additional expert witness. … Werecommend an independent, non-governmental selection panelwhich would liaise with relevant professional bodies to compile a listof possible experts from which the trial judge would make his or herselection. The appointments system would be transparent and would



78 It would be better if fundamental issues pertaining to the reliability of forensic science techniqueswere addressed by a well-resourced international body. For a variety of reasons, and withoutwanting to diminish the value and scope of international consensus and collaboration, thisparticular proposal is focused on enhancing domestic admissibility practice through timely adviceon reliability. International collaborations, in contrast, are difficult to set up, often procedurallystifling, vastly more expensive, consume much more time, and are inevitably subject to a range ofpolitical perspectives and compromises dictated by national interests.

79 Law Commission, Report, above n. 28 at para. 1.17.

ensure that any court-appointed expert is properly screened forexpertise and impartiality.80

Even in response to very complex expert disagreement in the most serious cases,recourse to impartial court-appointed experts appears to be ‘exceptional’ and theproposed procedure cumbersome.

Reliance upon impartiality is likely to be misleading.81 Impartiality—in terms ofattempts to be objective or unbiased—is generally desirable, but in criminal justicesettings will often be less important than substantial independence and obtainingevidence of reliability derived from empirical study.82 Recourse to expert impar-tiality, for reasons explained below, does not seem to have changed the culture ofexpert witnessing in the civil sphere or jurisdictions with formal codes of conductfor expert witnesses (for example, the courts of New South Wales). Indeed,witnesses proffering some of the most questionable opinions in English courtstend to be members of professional societies or institutions (for example, theForensic Science Society) that maintain formal codes that include proscriptions onpartisanship.

The meaning and significance of (apparent) impartiality is likely to be far moreambiguous than demonstrable evidence of reliability predicated on explicit criteria.For we can often represent the same evidence and practices as partial or impartialdepending on what we emphasise and how we understand and characteriseexpertise.83 Individuals with no obvious alignments or theoretical conceits are likelyto be considered impartial even if they hold unknown or undisclosed commit-ments, extreme or marginal views, or misrepresent (whether deliberately orinadvertently) evidence about their practices.84 The appearance of impartiality lies inthe eye of the beholder—largely dependent upon the knowledge and abilities of the



80 Ibid. at 179; see also para. 3.100 and cl. 3.81 Contrast modern studies of the sciences, e.g. S. Shapin, The Scientific Life: A Moral History of a Late

Modern Vocation (University of Chicago Press: Chicago, IL, 2008); H. Collins, Gravity’s Shadow: TheSearch for Gravitational Waves (University of Chicago Press: Chicago, IL, 2004); S. Shapin, ‘Cordelia’sLove: Credibility and the Social Studies of Science’ (1995) 3 Perspectives on Science 255; L. Daston andP. Galison, Objectivity (Zone Books: Boston, 2007).

82 Though it might, as in R v Wood [2012] NSWCCA 21, be used to interpret the performance of anexpert witness.

83 S. Yearley, ‘The Relationship between Epistemological and Sociological Cognitive Interests: SomeAmbiguities Underlying the Use of Interest Theory in the Study of Scientific Knowledge’ (1982) 13Stud Hist Phil Sci 353 at 375; S. Woolgar, ‘Interests and Explanation in the Social Study of Science’(1981) 11 Soc Stud Sci 365.

84 Undisclosed ‘conflicts of interest’ have created serious problems in biomedical research. See, e.g.,International Committee of Medical Journal Editors, Uniform Requirements for Manuscripts Submittedto Biomedical Journals: Writing and Editing for Biomedical Publication, §II.D (2005).

observer (and the context in which observation takes place).85 Moreover, anacademic witness called to present critical insights to assist the defence—such as arebuttal expert—might appear partisan, yet that appearance (or reality) may have notangible impact on their opinion or its significance.86 Historically, courts have notalways interpreted the failure of forensic scientists to consider hypotheses or eventsconsistent with non-guilt to constitute partisanship. Nor have they considered theactive involvement of forensic scientists in the investigation, including beingunnecessarily exposed to gratuitous information, or the failure to test a techniqueor ability, as detrimental to their implicit (or perceived) impartiality. Whenendeavouring to ascertain the value of a technique (and derivative opinion)evidence of actual ability and accuracy, derived from rigorous studies, is typicallymore informative than assertions about (im)partiality.87

In the Law Commission’s Report, impartiality seems to have more to do withgeneral honesty, such as revealing limitations and contrary opinions and awillingness to (speculatively) qualify the expression of opinions rather thanwhether the expert knows about or utilises techniques that are likely to reducetheir level of error or is involved in research that might actually identify or addresslimitations. This narrow approach to impartiality goes predominantly to credi-bility, whereas a broader approach substantially coincides with elements ofreliability. While impartiality may be a relatively weak epistemic indicator, weshould nevertheless expect the institutionalised and commercial forensic sciencesto be organised in ways that structurally remove as many potential partiality andbias problems, including conflicts of interest and exposure to gratuitous infor-mation, as possible. Conversely, in principle there are fewer reasons to be asconcerned about the partiality of (defence) rebuttal experts. Instead, we shouldfocus primarily on the substance of their evidence.88

A MAP does not require members to be or to appear impartial. Individual membersmay hold views, even strong views. Instead, a MAP requires members to be



85 R. Albury, The Politics of Objectivity (Deakin University Press: Maryborough, 1983).86 See Edmond and San Roque, above n. 6. More generally, see I. Mitroff, The Subjective Side of Science

(Elsevier: Amsterdam, 1974); M. Mulkay, ‘Norms and Ideology in Science’ (1979) 4 Soc Sci Inform 637.87 Although failure to adequately study a technique or ground an opinion could be used to suggest

partiality. Similarly, the failure to implement procedures for managing a range of contextualeffects and biases might also be used to suggest partiality and indifference. See I. Dror, D. Charltonand A. Peron, ‘Contextual Information Renders Experts Vulnerable to Making Erroneous Identifi-cations’ (2006) 156 Forensic Science International 74; D. Krane, S. Ford, J. Gilder et al., ‘SequentialUnmasking: A Means of Minimizing Observer Effects in Forensic DNA Interpretation’ (2008) 53Journal of Forensic Sciences 1006.

88 Law Commission, Report, above n. 28 at para. 3.93; see also Part 1, Section 2(e).

open-minded, even-handed and fair in the way they respond to evidence. Therange of individuals and disciplines will help to inform discussion and balanceviews and perspectives. With court-appointed, joint and party experts, by contrast,selection is everything. Moreover, members of the MAP are not paid (or dependentupon) fees and are, therefore, less likely to be influenced by the need to satisfyemployers, retain institutional standing, or secure future employment. Unlikeparty and even some court-appointed experts, the panel will have little invested inthe continued use of techniques (and theories).

I accept that court-appointed experts may offer valuable assistance in helping todecide individual cases.89 However, when it comes to assessing the reliability oftechniques, particularly those used or likely to be used routinely, other mecha-nisms involving wider collectives of experts seem more appropriate. The generalassessment of forensic science and medical techniques should not be dependenton an individual—however selected—and/or support adduced by the parties in aparticular adversarial setting. It is worth noting that the NAS report recom-mended establishing an independent body (a National Institute of ForensicSciences) responsible for research, standards and accreditation, rather thangreater recourse to impartial or court-appointed experts.90

Similarly, concurrent evidence—allowing the expert witnesses to testify togetherand comment upon each other’s reports and testimony—may have some utility incontested admissibility hearings.91 However, under FRE 702 (and Daubert andKumho) and the draft English Bill, the primary issue is reliability rather thanimpartiality or encouraging consensus from within a ‘field’ (or among a smallgroup of interested experts). Concurrent evidence has a tendency to bring‘experts’ from the same ‘field’ together, even where the existence of ‘the field’ orvalue of particular techniques and experience is controversial. Operationalised in



89 Though see the discussion of experience in the United States in J. Cecil and T. Willging,Court-Appointed Experts: Defining the Role of Experts Appointed under the Federal Rule of Evidence 706(Federal Judicial Centre: Washington, 1993).

90 National Research Council (NAS), above n. 31, Recommendation 1. Significantly, the NAS reporthas not had the kind of dramatic impact or influence that might have been anticipated—and seemsto have been expected by some members of the committee: H. Edwards, ‘Solving the Problems thatPlague the Forensic Science Community’ (2010) 50 Jurimetrics 5. It is important to recognise that thenow defunct Forensic Science Service (UK) and the Australian National Institute of ForensicSciences (NIFS) are substantially different to the research-oriented, scientific institute proposed inthe NAS report.

91 G. Edmond, ‘Merton and the Hot Tub: Scientific Conventions and Expert Evidence in AustralianCivil Procedure’ (2009) 72 L & Contemp Probs 159; G. Edmond, ‘Impartiality, Efficiency orReliability? A Critical Response to Expert Evidence Law and Procedure in Australia’ (2010) 42Australian Journal of Forensic Sciences 83.

this way, the individuals participating in concurrent evidence sessions might notidentify the kinds of limitations and concessions that are appropriate—especiallyif it might undermine their legal authority.92

While the MAP also seeks to generate consensus, unlike concurrent evidence theparties do not select the experts and any consensus (or disagreement) transcends‘the field’. More importantly, the focus of the MAP is explicitly on the reliability oftechniques and derivative opinions. The MAP brings a wide spectrum of expertiseto assist with the assessment of reliability. In this way, the advice of the MAP isneither parochial nor professionally invested in securing or maintaining admissi-bility for particular techniques and opinions. The MAP is generally indifferent tothe existence (or non-existence) of fields and ought to be sceptical about specialabilities that are not empirically supported.

7. The elephant in the courtroom: scientific and technical literacy

Notwithstanding concessions concerning the admission of too much insuffi-ciently reliable expert opinion evidence and weak trial safeguards, the LawCommission Report remains (arguably over-) confident about the currentjudiciary and its ability to change. Questions about the abilities of lawyers, judgesand jurors to obtain, manage and understand expert opinion evidence in thecontext of the accusatorial trial remain largely unanswered.93

The issue of scientific and technical competence tends only to appear on themargins of the Report, primarily in submissions from non-legal organisations:

Practical objections were raised by the Forensic Science Service …because of the possibility of repeated challenges against evidence and,in their view, the difficulty of providing the judiciary ‘with sufficientscientific knowledge’ for judges to be able properly to assess thereliability of forensic scientific evidence. Similarly, the British Psycho-logical Society … doubted whether it would be practicable for judges



92 Some types of expert (or communities), for example, may not even be conversant with relevantmethodological and statistical issues or may have shared interests in not disclosing them.

93 See, e.g., M. Kovera and B. McAuliff, ‘The Effects of Peer Review and Evidence Quality on JudgeEvaluations of Psychological Science: Are Judges Effective Gatekeepers?’ (2000) 85 Journal of AppliedPsychology 574; J. Cooper, E. A. Bennett and H. L. Sukel, ‘Complex Scientific Testimony: How DoJurors Make Decisions?’ (1996) 20 Law and Human Behavior 379; J. Groscup et al., ‘The Effects ofDaubert on the Admissibility of Expert Testimony in State and Federal Criminal Cases’ (2002) 8Psychology, Public Policy & Law 339; C. Guthrie, J. Rachlinski and A. Wistrich, ‘Inside the Judicial Mind’(2001) 86 Cornell L Rev 777.

to acquire sufficient knowledge to make informed rulings on thereliability of expert opinion evidence.94

These issues and their unsettling implications are never pursued. Rather, theReport and Consultation Paper maintain confidence in judges and judicialabilities, despite the fact that judges have continuously allowed unreliable andspeculative expert opinion evidence to contaminate criminal proceedings sincethe origins of the forensic sciences more than a century ago.

Unsupported confidence in judicial abilities is maintained throughout the Report.Implicitly, all that is required to address the current malaise are revised standardsand some limited training to give lawyers and judges the requisite abilities toassess expert opinion evidence properly—whatever the subject-matter and howevercontroversial. The Consultation Paper and Report both suggest that judges do notneed to become—what Chief Justice Rehnquist described, in his Daubert dissent,as—‘amateur scientists’:

In this context it is worth repeating a comment we set out in ourconsultation paper with reference to evidence of a scientific nature:‘judges do not need to be trained to become scientists, they [merely]need to be trained to be critical consumers of the science that comesbefore them’.95

Rather, through training and judicial education, lawyers and judges are tobecome ‘critical consumers’ of science.96 However, just how this will work and thenature of the proposed curriculum remain disappointingly opaque.97



94 Law Commission, Report, above n. 28 at para. 3.20.95 Law Commission, Report, above n. 28 at paras. 5.112 and 8.8; Law Commission, Consultation Paper,

above n. 28 at 44, 46. Quoting S. Gatowski et al., ‘Asking the Gatekeepers: A National Survey ofJudges on Judging Expert Evidence in a Post-Daubert World’ (2001) 25 L & Hum Behav 433. CompareG. Edmond, ‘Judging Surveys: Experts, Empirical Evidence and Law Reform’ (2005) 33 Fed L Rev 95.

96 They will be able: ‘properly to investigate and determine evidentiary reliability’ and determine ‘ifthere is a plausible basis for doubting its reliability’ (Law Commission, Report, above n. 28 at paras. 5.83,5.106; italics added). For a more theoretically informed, though controversial, discussion, see H.Collins and R. Evans, Rethinking Expertise (University of Chicago Press: Chicago, Il, 2007).

97 Though the Report (at para. 3.53) does suggest that the Federal Judicial Center’s Reference Manual onScientific Evidence, 2nd edn (2011) might be a useful example. The Consultation Paper, in particular,was hostile to perspectives from the history and philosophy of science and science and technologystudies. This is unfortunate because these are some of the disciplines best positioned to helplawyers and judges understand expertise. See e.g. E. Hackett, O. Amsterdamska, M. Lynch and J.Wajcman (eds.), The Handbook of Science and Technology Studies, 3rd edn (MIT Press: Cambridge, MA,2008); S. Jasanoff et al., Handbook of Science and Technology Studies, 2nd edn (1995); H. Collins and T.Pinch, The Golem: What Everyone Should Know about Science (Cambridge University Press:Cambridge,1993); S. Yearley, Making Sense of Science (Sage: London, 2005).

The failure of overarching criminal justice principles and emerging, ifoccasionally inconclusive, empirical research to shape admissibility practicesuggests that any training (or re-education) might have to overcome a range ofdeeply held professional and ideological commitments and institutionalpressures. Empirical insights from North America and the recent history ofpractice in England and Wales suggest that judges will have difficulties concedingthat many long-held beliefs about the criminal trial and its safeguards, the valueof many forensic science techniques, and the abilities of jurors (within contem-porary trial settings) might be mistaken (see Part 1, Section 2(b)).98 Becauseprofessional experience, ideology and institutional pressures will influence howany new admissibility criteria are applied, unless re-education and reform isbroadly conceived and more persuasively presented, any putatively moredemanding admissibility criteria are likely to be less effective than imagined.99

Furthermore, we should not overlook the fact that many of the problems withexpertise are embedded in other parts of the criminal justice system—with thepolice, institutionalised forensic science and medicine, and prosecutors, defencelawyers and jurors.100

Too much controversial expert opinion evidence would seem to be given to juriesto adjudicate.101 A rational process—particularly a process that purports to befair—cannot place blind faith in a judge (let alone a lay jury) to resolvecomplex or subtle technical questions, especially where the requisite infor-mation is not provided.102 Principled deference on the question of reliability, andthe exclusion of insufficiently reliable techniques and opinions, seem like areasonable compromise between varying degrees of technical (in)competence



98 See also G. Edmond and K. Roach, ‘A Contextual Approach to the Admissibility of the State’sForensic Science and Medical Evidence’ (2011) 61 University of Toronto LJ 343; Edmond, above n.31; Edmond and San Roque, above n. 6.

99 Limited judicial education is unlikely to produce the kinds of cultural and institutional changesrequired. This is, perhaps, one benefit obtained from involving an independent panel ofgeneralists, not steeped in legal mythologies, the experience of judges, or beholden to pastpractice. Not insignificantly, the non-systematic experience of judges raises similar issues to theuse of experience as the basis for conclusions among forensic scientists.

100 On problems with the cultures of the forensic sciences, see J. Mnookin et al., ‘The Need for aResearch Culture in the Forensic Sciences’ (2011) 58 UCLA L Rev 725; S. Cole, ‘AcculturatingForensic Science: What Is “Scientific Culture”, and How Can Forensic Science Adopt it?’ (2010) 38Fordham Urban LJ 435. See also Law Commission, Report, above n. 28 at para. 3.34.

101 Such concerns are long-standing, e.g. L. Hand, ‘Historical and Practical Considerations RegardingExpert Testimony’ (1901) 15 Harv L Rev 40.

102 See generally, E. Beecher-Monas, Evaluating Scientific Evidence: An Interdisciplinary Framework forIntellectual Due Process (Cambridge University Press: New York, 2007); S. Brewer, ‘Scientific ExpertTestimony and Intellectual Due Process’ (1998) 107 Yale LJ 1535.

and lay participation.103 A MAP has the ability to improve the quality of evidenceadmitted in criminal proceedings without displacing the democratic potentialof legal institutions.104

8. Conclusion: Modest law reform, principled deference and modular

expertise

Positioning the MAP as an adjunct to legal practice, making any reference anindicative review of existing studies and literatures, and limiting the advice toreliability rather than admissibility—thereby maintaining the dominant positionsof the trial judge (as gatekeeper) and the jury (as the tribunal of fact)—retains scopefor non-technocratic assessment of expertise and a continuing ability to challengetechniques and knowledge products in the quasi-public sphere of the trial (andappeal).105 By assisting courts with reliability determinations, the MAP representsrelatively modest law reform designed to make practice more consistent withformal rules and overarching legal principle.

In conclusion, it is my intention to situate the MAP in its broader socio-legalcontext. First, to reinforce the idea that the advice of a MAP is intended to improvethe quality of admissibility decision-making and legal outcomes without bindingthe hands of judges. Then, the final sub-section offers a partial theorisation of theMAP. Rather than invoke idealised (or romantic) models of science and technologyit suggests that legal institutions and systems (for example, criminal, civil, intel-lectual property, regulatory) should be developing models of expertise (andrelated admissibility criteria) that are conducive to their particular values, aspira-tions and social purposes.

(a) Principled deference and reasoned derogation

Imposing advice from a MAP on accusatorial proceedings does not dramaticallyinterfere with party discretion or prevent scope for public contests around admis-sibility and the meaning of reliability. Moreover, the MAP is not purporting todefine or characterise science (or expertise) for all times and circumstances—noteven for all criminal proceedings and investigative purposes. Instead, itsoperation is indexed to reliability and implicitly admissibility decision-making inthe criminal justice system. It represents an intervention, really a supplement,that might not be entirely unwelcome by the parties or the decision-makers. Givenvery permissive admissibility standards in all common law jurisdictions, defence



103 Contra Collins and Evans, above n. 96.104 Jasanoff, above n. 9; S. Jasanoff, Designs on Nature (Princeton University Press: Princeton, NJ, 2005).105 A. Irwin and B. Wynne (eds.), Misunderstanding Science (Cambridge University Press: Cambridge,

1996).

lawyers are unlikely to resist the introduction of a MAP—even if its advice is applied(symmetrically) to expert opinion evidence adduced on behalf of the accused.106

Prosecutors might resent more rigorous enforcement of admissibility standards,but their interest in rectitude and fairness, as ‘ministers of justice’, should tempersuch antipathies. Trial and appellate judges dealing with novel proffers, contro-versial techniques and techniques that are not controversial but should be arelikely to welcome guidance from beyond the parties and their experts.

Through published advice, the MAP is intended to assist courts with admissibilitydecisions. The tremendous variety of scientific, technical and medical evidenceand the diverse ways in which they are used mean that there could only be advicedirectly relevant to a small percentage of contested admissibility hearings. Asan institution, the MAP’s scope for direct contributions would seem to beconstrained. It certainly would not dominate all legal discussion and jurispru-dence pertaining to the admissibility of expert opinion evidence. In manycircumstances any advice would be indicative or merely illuminative. It might,nevertheless, exert a disciplining influence in controversial areas and, over time,on more general legal responses to scientific and technical forms of knowledgeand expertise.

In the end, the MAP’s exemplary response and advice on techniques would not bedeterminative. It cannot prevent admission, further research, new evidence, oranticipate how techniques might develop or be contested (and defended) in futureadmissibility contests in and out of courts. The model facilitates principled deferencewhile providing scope to reject or qualify the advice on any particular topic—thatis, reasoned derogation. The advice of the MAP should be sufficiently grounded torequire any attempt to circumvent, displace or qualify it to be compelling—legallyand/or epistemologically. Reasoned, or principled, derogation is always availablebut warrants some persuasive rationalisation linked to the technique and theevidence supporting it. The strength of the case against the accused, decisions byother courts, and the length of time since critical advice was produced do not, ofthemselves, provide evidence of reliability (or grounds for admitting techniques).

(b) More modular expertise

This article has endeavoured to sketch an alternative—though really supple-mentary—approach to assessing the admissibility of expert opinion evidence insome criminal proceedings. Implicitly, it argues that legal models of expertiseshould be shaped primarily by legal principle and system goals. That is, theyshould be more modular, indexed to: the participants; the type of proceeding; the



106 On (a)symmetry, see Part 1, Section 2(e); Edmond and Roach, above n. 98.

type of expertise; the broader socio-political purposes of different types of institu-tions and activities, as well as what we actually know about technical capabilities.

Once we recognise that expertise can be described in a variety of non-essentialways we can begin to reflect upon attributes that might be considered desirable inparticular legal contexts. That is, we can begin to think about tailoring models ofexpertise and admissibility criteria to achieve or uphold specific socio-legal valuesand objectives. By way of example, the model of expertise governing admissibilityand proof might be quite different in a system that values crime control overaccuracy and/or fairness. Similar considerations arise in the civil sphere when wefactor in the resources of the parties and a range of regulatory aspirationsassociated with risk, deterrence and compensation.107

A demanding model of expertise—one, for example, that inflexibly incorporatesmost (or all) of the Daubert criteria or their proposed English ‘equivalents’ (see Part1: Sections 1 and 2(b))—might be desirable when determining the admissibility andweight of a type of incriminating expert opinion evidence that is routinelyadduced by the state. Forensic science techniques, particularly those usedroutinely, should be able to satisfy quite onerous admissibility criteria. Notbecause these criteria embody authentic (or proper or good or real and so on)science or its essential attributes, or even reliability for all times and places, butrather because legal values, the epistemic weakness of many forensic sciences, andemerging evidence about the frailty of the adversarial trial all suggest thatunreliable expert evidence subverts criminal trial principles and objectives.108 Inthe context of criminal justice, imposing (and enforcing) reliability-based admissi-bility standards fulfils a pragmatic function more consistent with the premiumplaced on factual rectitude and the practical difficulty of overcoming incrimi-nating, but unreliable, expert opinion evidence.

Onerous admissibility criteria might be less well suited to civil proceedings andregulatory settings. Where, for example, an impecunious plaintiff alleges that apharmaceutical caused a debilitating injury, it might be inappropriate to burdenthat individual (or ‘group’) with proof if the drug has not been adequately tested,or only tested by private research organisations engaged by the manufacturer



107 C. Cranor, Toxic Torts: Science, Law and the Possibility of Justice (Cambridge University Press:Cambridge, 2006); M. Berger, ‘Eliminating General Causation: Notes Towards a New Theory ofJustice and Toxic Torts’ (1997) 97 Columbia Law Rev 2117.

108 See G. Edmond and D. Mercer, ‘Experts and Expertise in Legal and Regulatory Settings’ inG. Edmond (ed.), Expertise in Regulation and Law (Ashgate: Aldershot, 2004); D. Caudill and L. LaRue,No Magic Wand: The Idealization of Science in the Law (Rowman & Littlefield: Lanham, MD, 2006).

through clinical trials designed to facilitate regulatory approval.109 Strictlyapplying Daubert-style reliability criteria and privileging pre-litigation research,for example, makes it difficult for plaintiffs to prove injury and loss.110 It mightalso undermine civil justice and regulatory objectives such as deterring poten-tially dangerous activities and products as well as provide manufacturers withincentives not to sponsor independent safety and efficacy studies. Simulta-neously, it transfers the risks of uncertainty, inaction, under-compensation andmalfeasance from manufacturers (and employers) to a range of other publics. Byaccepting uncritically the nature of science, or treating definitions of expertise aseffectively neutral, many of these important (and contested) issues from our civil,criminal and regulatory systems risk elision or displacement.111

Unfortunately, judges and commentators have tended to focus on the natureof science and (what are presented as) its essential and enduring features—anddistractions such as partisanship—rather than think about what differentbodies of law are intended to achieve, the kinds of values and risks at stake, andhow admissibility standards might be designed and practically operation-alised.112 Perhaps unwittingly, judges and legal commentators have tended toembrace models of science and expertise that are reductionist.113 Reveal-ingly, these models have not been applied consistently. In most jurisdictions—though most conspicuously in the United States where civil juries are in



109 See discussion and references in G. Edmond, ‘Judging the Scientific and Medical Literature: SomeLegal Implications of Changes to Biomedical Research and Publication’ (2008) 28 Oxford J of LegalStudies 523.

110 See Daubert v Merrell Dow Pharmaceuticals Inc., 43 F3d 1311 (9th Cir 1995), discussed in G. Edmond,‘Supersizing Daubert: Science for Litigation and its Implications for Legal Practice and ScientificResearch’ (2007) 52 Villanova Law Rev 857.

111 Compare P. Huber, Galileo’s Revenge: Junk Science in the Courtroom (Basic Books: New York, 1991) andN. Foster and P. Huber, Judging Science (MIT Press: Cambridge, MA, 1997) with W. Wagner and R.Steinzor (eds.), Rescuing Science from Politics: Regulation and the Distortion of Scientific Research(Cambridge University Press: Cambridge, 2006); Project on Scientific Knowledge and PublicPolicy, Daubert: The Most Influential Supreme Court Ruling You’ve Never Heard of (Tellus Institute:Boston, MA, 2003); C. Cranor, Toxic Torts (Cambridge University Press: Cambridge, 2007); L. Finley,‘Guarding the Gate to the Courthouse: How Trial Judges Are Using their Evidentiary ScreeningRole to Remake Tort Causation Rules’ (1999) 49 DePaul L Rev 335.

112 See, e.g., M. Michael, ‘Lay Discourses of Science: Science-in-general, Science-in-particular, andSelf’ (1992) 17 Science, Technology and Human Values 313.

113 Many amici and commentators characterised the Daubert criteria, particularly testing/falsification, as an accurate characterisation of scientific practice. Compare: H. M. Collins,Changing Order (University of Chicago Press: Chicago, IL, 1986); S. Shapin, Never Pure: HistoricalStudies of Science as if it Was Produced by People with Bodies, Situated in Time, Space, Culture, andSociety, and Struggling for Credibility and Authority (Johns Hopkins University Press: Baltimore, MD,2010).

widespread use—judges have applied universal admissibility criteria in ways thatappear to be markedly shaped by a range of institutional and ideological consider-ations.114 Against principle, admissibility standards tend to be applied moreaggressively in civil proceedings (against expert opinion evidence adduced byplaintiffs) and more leniently in criminal proceedings—generally accommo-dating incriminating expert opinion evidence adduced by the state (see Part 1,Section 2(b)).115

Rejecting a vision of science (or expertise) with fixed and essential features opensthe possibility of tailoring institutions, knowledge production and adjectival rulesand practices to particular contexts of use.116 By framing knowledge needs in thisway we can appeal to technical abilities though without abandoning public inputand without abrogating or denying the role of socio-political factors in knowledgeconstruction, legal practice, and the maintenance of institutional legitimacy andsocial order.117 Adopting a more modular approach to definitions of science andexpertise has the potential to improve system performances in ways that areconsistent with legal principle. More modular approaches can be developed inways that are sensitive to the distribution of resources and expertise, ethical andlegal obligations, as well as risks.

Ultimately, our criminal justice institutions have to decide if they truly valuetraditional legal principles—such as the presumption of innocence, placing theburden of proof on the state, excluding unfairly prejudicial evidence,conducting a fair proceeding, requiring proof beyond reasonable doubt, and notconvicting the innocent. If so, they should be willing to reform legal practicein order to uphold these principles and exclude insufficiently reliableexpert opinion evidence. Those administering trials should be genuinely inter-ested in the quality of incriminating expert opinion evidence and the actualability of the system to cope with it. Whether our legal institutions are up tosuch responsibilities and whether various publics are capable of effectively



114 W. Haltom and M. McCann, Distorting the Law: Politics, Media, and the Litigation Crisis (University ofChicago Press: Chicago, Il, 2004); E. Cheng and A. Yoon, ‘Does Frye or Daubert Matter? A Study ofScientific Admissibility Standards’ (2005) 91 Virginia L Rev 471.

115 G. Edmond and D. Mercer, ‘Daubert and the Exclusionary Ethos: The Convergence of Corporateand Judicial Attitudes towards the Admissibility of Expert Evidence in Tort Litigation’ (2004) 26Law & Policy 231.

116 Edmond and Roach, above n. 98.117 Explicitly modular approaches to expertise have the potential to make some of the debates

around legal knowledges and co-production more transparent. See Jasanoff, above n. 47; G.Edmond, ‘The Law-Set: The Legal-Scientific Production of Medical Propriety’ (2001) 26 Science,Technology & Human Values 191.

participating, even within reformed institutional arrangements—of which theMAP is merely one possibility—are open to doubt. Our criminal justice rules andprocedures do, however, deserve serious study and refinement before moredrastic reforms are imposed by those less persuaded of the value of traditionallegal principles.



ADVICE FOR THE COURTS? SUFFICIENTLY RELIABLE ASSISTANCE WITH FORENSIC SCIENCE Advice for the courts?...

Documents

Transcript of ADVICE FOR THE COURTS? SUFFICIENTLY RELIABLE ASSISTANCE WITH FORENSIC SCIENCE Advice for the courts?...