Empirical Research on Deliberative Democracy: A

Review of the Literature

Abstract

This paper reviews the recent literature on empirical

research into deliberative decision-making. Following

Thompson, it distinguishes between the conceptual criteria,

evaluative standards and empirical conditions associated

with good or effective deliberation. It finds that the

question of empirical conditions has been approached from

three distinct directions. In one stream in the literature,

which it calls the behavioural-procedural stream, work has

focused on producing an empirical definition of

deliberation which permits an assessment of the degree to

which deliberation is occurring. A second stream in the

literature, the feasibility-capacity stream, has involved seeking

evidence to support (or undermine) the claims of normative

theory about deliberation as a social process. A third

stream, the institutional-environmental or boundary conditions

stream, involves considering the effect on discursive

practice of a series of contingent factors external to the

process of deliberation itself. Notable by its almost

complete absence, however, is any work that aims to

evaluate the technical effectiveness of deliberative

policy- and decision-making relative to other types of

process, democratic or otherwise.

Introduction

In this paper we review a selection of the recent

literature reporting the results of empirical research into

deliberative decision-making. This current of research

appears to be a response to the increasing incidence of and

interest in participatory approaches to governance and

policy decision-making. These approaches are in turn a

response to the perception that existing modes of social

and political governance lack legitimacy. Hartz-Karp and

Briand, for example, argue that in advanced societies,

the public has unquestioningly handed over much of the

necessary decision-making to ‘experts’, to whom

implicitly we have assigned the ability to determine

what counts as knowledge and what does not. Thus

2

policy-makers too readily accept their own views as

sound, but treat the views of ordinary citizens – even

in regard to matters properly within their realm of

‘expertise’, such as values and priorities – as mere

‘preference’ and ‘opinion’’1

In a similar vein, Wagenaar suggests that

the increasing technical complexity of societal

sectors, in combination with a complex cross-current of

administrative ideologies that favor expertness,

managerial efficacy, and allocative efficiency, has

resulted in rather a high-handed, technocratic style of

policy making in which administrators and external

experts claim the right to manage entire societal

sectors and in which the citizens who are at the

receiving end of these policies are consequently

disenfranchised from the governance of their own

environment.2

Political participation can take a dizzying array of

different forms, but Huitema and his colleagues have

3

usefully parsed these into those arising from a pluralist

discourse and those that emerge from the discourse of

deliberation.3 The emphasis in the former case is on

participation as a means of guaranteeing the effective

representation of individuals and groups in a political

process based on balancing competing values and interests.

In the latter case, by contrast, the emphasis is on the

identification and articulation of a single authentic

collective interest – Rousseau’s ‘general will’ as opposed

to the mere ‘will of all’ – and the assembly of a reasoned

argument for the pursuit of a particular course of social

or collective action in the light of that shared interest.4

According to Baccaro & Papadakis, the principal claim of

the advocates of deliberative participation is that it

‘produces not just a richer texture of democracy but also

more effective public policy’.5 One might add that,

particularly for those authors who adopt broadly 1 Janette Hartz-Karp & Michael K. Briand, ‘Institutionalizing

deliberative democracy’, Journal of Public Affairs, 9(2) (2009), p.135-6.

2 Hendrik Wagenaar, ‘Governance, Complexity, and Democratic

Participation’ American Review of Public Administration, 37(1) (2007) 17-50,

p.22.4

Habermasian theoretical approaches, the increased

effectiveness of policy outcomes is held to be a direct

result of the democratic nature of the decision-making

process.

While there is some doubt about whether a sharp distinction

between pluralist and deliberative forms of participation

can be maintained in practice, in this review we will

nevertheless confine ourselves to considering those forms

of democratic participation that, at least as a regulatory

ideal, aim to seek consensus not just on action but on the

technical and normative reasons for action. The books and

papers referenced in the review were identified via a trawl

of the websites of the main academic research centres and 3 David Huitema, Marleen Kerkhof & Udo Pesch, ‘The nature of the

beast: are citizens' juries deliberative or pluralist?’ Policy Sciences,

40 (2007), 287-311.

4 Mansbridge uses the terms ‘adversary’ and ‘unitary’ democracy in

drawing essentially the same distinction. See Jane Mansbridge, Beyond

Adversary Democracy, (Chicago: University Of Chicago Press, 1980).

5 Lucio Baccaro & Konstantinos Papadakis, ‘The downside of

participatory-deliberative public administration’, Socio - Economic

Review, 7(2) (2009), p. 246.5

principal figures known to be involved in work on

deliberative democracy, together with a closely-focused

search of electronic journals.6 The search requested

articles with the words ‘deliberation’ or ‘deliberative’

together with the word ‘empirical’ in the title or

abstract. In the end a total of around 75 journal articles

and working papers were reviewed. The relatively narrow

overall search strategy means that most work on non-

deliberative forms of participatory governance and

democracy is likely to have been excluded.

What is Deliberative Democracy?

Thompson suggests that there are three elements in the

analysis of political deliberation that need to be

distinguished: conceptual criteria, evaluative standards,

and empirical conditions. ‘Conceptual criteria stipulate

what is necessary for a practice to count as deliberation.

Evaluative standards specify what counts as good (or

better) deliberation. Empirical conditions indicate what is

necessary for producing good deliberation (or less

6 The literature search was carried out in July 2009.6

strongly, what may contribute to producing good

deliberation)’.7 We will use Thompson’s analytical model to

structure the discussion that follows.

Conceptual Criteria

While, as Rosenberg points out,8 work in the anglo-american

political science tradition, particularly that of John

Rawls, has been of great significance in the development of

work on deliberative democracy, the writing of Jürgen

Habermas seems to have been the most important catalyst for

the increasing interest in deliberation apparent from the

mid-1990s. This is reflected in the relatively narrow range

of conceptual definitions of deliberative democracy that

feature in the literature. As Schneiderhan and Khan point

out,9 almost without exception these definitions focus on

reason-giving and inclusiveness, thereby taking up the essentials

of Habermas’s concept of communicative action and his

discourse principle. Thus normative theories of

deliberative democracy emphasize communication about

7 Dennis F. Thompson, ‘Deliberative Democratic Theory and Empirical

Political Science’ Annual Review of Political Science, 11(1) (2008), 497-520,

p.501.7

preferences rather than the aggregation of fixed preferences

via individual choice or voting.10 Normative theory also

proposes that ‘democratic legitimacy resides in the right,

ability and opportunity of those subject to a collective

decision to participate in deliberation about the content

of that decision’.11 Although there no suggestion in the

literature that inclusiveness is the less important of the

two principles, it is certainly less uniquely

characteristic of deliberation than reason-giving. As

Thompson puts it, ‘the most important distinguishing

characteristic of deliberation, is mutual justification –

presenting and responding to reasons intended to justify a

political decision’.12

Evaluative standards

In the literature the primary evaluative standards applied

to reason-giving are:

• that reasons are couched in terms of the common good,

including the good that arises from permitting

individuals or groups to pursue their particular

8

interests in certain circumstances;

• that reasons are used sincerely and/or truthfully; and

• that there is a logical relation between a given reason

and the preference it supports.13

8 Shawn Rosenberg, ‘An Introduction: Theoretical Perspectives and

Empirical Research on Deliberative Democracy’ in Shawn Rosenberg,

ed., Deliberation, Participation and Democracy: Can the People Govern? (London:

Palgrave Macmillan, 2007).

9 Erik Schneiderhan, & Shamus Khan, ‘Reasons and Inclusion: The

Foundation of Deliberation’, Sociological Theory, 26(1) (2008), 1-24.

10 André Bächtiger, Marco Steenbergen & Axel Tschentscher, ‘Developing

deliberative democracy: A research report and a research agenda’,

University of Berne Institute for Interdisciplinary Deliberation

Studies (2008) Available at:

http://www.bids.unibe.ch/unibe/rechtswissenschaft/oefre/bids/content

/e3409/e3822/e3824/linkliste3826/

Bchtiger:Steenbergen:Tschentscher.pdf [Accessed May 12, 2009].

11 John Dryzek, ‘Democratization as deliberative capacity building’,

University of Berne Institute for Interdisciplinary Deliberation

Studies (2008), p 3. Available at:

http://www.bids.unibe.ch/unibe/rechtswissenschaft/oefre/bids/content

/e3409/e3822/e3824/linkliste3831/Dryzek.pdf [Accessed May 12, 2009].9

As applied to the concept of inclusiveness, we find

standards that evaluate not so much whether the relevant

subjects or their representatives are formally involved in

the deliberative process, but the degree to which

participants in deliberation are genuinely able to

contribute to the discussion and to have their voices

heard. Thus we have the standards of:

• participation in the sense of the availability of

substantive opportunities to contribute to the

discussion;

• respect for the arguments and the perspective of other

12 Thompson, ‘Deliberative Democratic Theory and Empirical Political

Science’, p.504.

13 See for example Jürg Steiner, André Bächtiger, Markus Spörndli &

Marco Steenbergen. Deliberative Politics in Action: Analysing Parliamentary Discourse

(Cambridge: Cambridge University Press, 2004); Wagenaar,

‘Governance, Complexity, and Democratic Participation’; Robert E.

Goodin, ‘Sequencing Deliberative Moments’ Acta Politica, 40(2) (2005),

182-196. Dryzek, ‘Democratization as deliberative capacity

building’; Tali Mendelberg & John Oleske, ‘Race and Public

Deliberation’, Political Communication, 17 (2000), 169-191.10

participants; and

• openness to changing preferences in response to the force

of the better argument.14

In truth this last standard applies to both reason-giving

and inclusiveness. Reason-giving obviously loses its

effectiveness as a means of reaching agreement the more

unwilling participants are to be persuaded away from their

starting positions. It is also true that it is pointless to

include a wide range of participants in a deliberative

process if there is no possibility that they might change

their minds in response to the different perspectives 14 See for example André Bächtiger & Jürg Steiner, ‘Introduction’, Acta

Politica, 40(2) (2005), 153-168; Christer Karlsson, ‘Deliberation at

the European Convention: The Final Verdict’, European Law Journal, 14(5)

(2008), 604-619; Simon Niemeyer, S., Selen Ayirtman & Janette Hartz-

Karp, ‘Achieving Success in Large Scale Deliberation: An Analysis of

the Fremantle Bridge Community Engagement Process’, Australian

National University (2008), Available at:

http://deliberativedemocracy.anu.edu.au/Frembridge/FremBridgeRpt.pdf

[Accessed May 11, 2009]; Michael Neblo, ‘Thinking through Democracy:

Between the Theory and Practice of Deliberative Politics’, Acta

Politica, 40(2) (2005), 169-181.11

brought to the discussion by others.

Empirical conditions

The question of the empirical conditions associated with

good or effective deliberation has been approached from

three distinct directions. In one stream in the literature,

which we can call the behavioural-procedural stream, work has

focused on producing an empirical definition of

deliberation which permits an assessment of the degree to

which deliberation is occurring. The conditions specified

tend to be a fairly straightforward operationalisation of

the evaluative standards we have just discussed, and thus

focus on the attitudes and discursive behaviour of

participants as well as the logic and structure of the

arguments they propose. In this stream, the possibility of

deliberation as a distinct mode of discursive interaction

is not in question. Rather, specifying the empirical

conditions for deliberation is merely an intermediate step

towards answering more substantive questions about the

outcomes and effects of deliberation.

12

A second stream in the empirical research literature, the

feasibility-capacity stream, has involved seeking evidence to

support (or undermine) the claims of normative theory about

deliberation as a social process. Researchers contributing

to this stream do not assume that deliberation is always

(or, in some cases, ever) possible, and tend to be

concerned with the social psychology of discursive practice

and with power relations arising from differences of class,

sex and ethnicity. They are interested in whether

supposedly deliberative interactions have the claimed

effect – consensus, legitimacy and so on – as well as the

extent to which the effects of such interactions can be

traced to social and psychological processes that are

coherent with normative theory.

A third stream in the empirical literature, the institutional-

environmental or boundary conditions stream, is slightly

different in that it overlaps with both of the other

approaches and is potentially coherent with either. This

approach involves considering the effect on discursive

practice of a series of contingent factors external to the

13

process of deliberation itself.

The behavioural-procedural stream

The discursive behaviour of participants has been an

important focus of empirical research on deliberation that

aims to assess the ‘deliberativeness’ of existing political

institutions and processes. For example, Holzinger has

analysed the debate in the German parliament on the

contentious issue of stem cell research.15 Her work, based

on the analysis of transcripts of the sessions in question,

seeks to assess the relative importance of arguing and

bargaining in the debate. This latter distinction is based

on J L Austen and John Searle’s work on speech acts.

Arguing involves speech acts such as claiming,

establishing, assuming, asking, justifying, contradicting

and judging, whereas bargaining involves an entirely

different set of discursive practices like demanding,

offering, promising, threatening, accepting and rejecting.

What Holzinger’s analysis shows, however, is that this

15 Katharine Holzinger, ‘Context or Conflict Types: Which Determines

the Selection of Communication Mode’, Acta Politica, 40(2) (2005), 239-

254.14

distinction is not adequate to the identification of

deliberative behaviour. As she herself concedes, there was

a great deal of arguing and very little bargaining in the

debate, as would be expected from the nature of the issue

in question, but very little of this argument was

deliberative. Rather, non-dialogical rhetoric was the

dominant form of speech, no consensus was reached and the

issue was ultimately resolved by voting.

Conover and Searing come at the question of identifying

deliberation rather more directly in their analysis of

political talk.16 Analysing the transcripts of focus groups

and, less conventionally, the text of letters published in

British newspapers, the authors seek to assess the extent

to which political talk satisfies their version of the

empirical conditions for deliberation: reciprocity,

publicity, non-tyranny and equality. They conclude that in

general, this kind of talk falls well short of the

deliberative ideal: ‘Although citizens are committed to

16 Pamela J. Conover & Donald D. Searing, ‘Studying ‘Everyday

Political Talk’ in the Deliberative System’, Acta Politica, 40(3)

(2005), 269-283.15

reciprocity in principle, in practice they sometimes find

it difficult to respect their fellow citizens. Public

discussions occur infrequently, for most everyday talk is

relatively private in nature. Finally, while everyday talk

typically involves weak contestation, it also reflects

inequalities in society, particularly when political

discussion becomes more public’.17

Perhaps the best-known attempt to provide an empirical

‘test’ for deliberation is the work of Steiner and his

colleagues at the University of Berne.18 This work, which

aims to evaluate the quality of debate in national

legislatures with respect to the evaluative standards of

deliberation, revolves around the ‘discourse quality index’

or DQI. The DQI provides a measure of the deliberativeness

of the discursive behaviour of legislators along the four

17 Conover & Searing, ‘Studying ‘Everyday Political Talk’ in the

Deliberative System’, p. 278.

18 Steiner et al., Deliberative Politics in Action: Analysing Parliamentary Discourse;

André Bächtiger, Markus Spörndli, Marco Steenbergen & Jürg Steiner,

‘The Deliberative Dimensions of Legislatures’, Acta Politica, 40(2)

(2005), 225-238.16

principal dimensions of participation, justification

(divided into level and content of justification), respect

(divided into respect for other groups, respect for group

interests and demands and respect for counterarguments) and

constructive politics (consensus-building). Participation

reflects the opportunity afforded to legislators to speak

without interruption. Justification is a measure of the

sophistication with which arguments are grounded and of the

degree to which arguments are couched in terms of the

common good as opposed to group interests. Respect measures

the degree to which participants’ interventions reflect

negative or positive attitudes to opposing groups,

interests and arguments. Finally, constructive politics is

a measure of the extent to which legislators make efforts

to find universally acceptable solutions as opposed to

simply trying to insist that their particular view prevail.

The DQI framework is arguably the most methodologically

sophisticated approach to an operationalization of the

evaluative standards for deliberation, despite its

(deliberate) omission of any means of assessing the

17

sincerity or truthfulness of participants. However, even if

it were possible effectively to address this lacuna, the

DQI would remain a tool of limited empirical value because

in itself it can shed no light on two crucial issues:

first, whether high-quality deliberation has a greater

tendency to produce policy consensus; and, second, whether

the policies and decisions that emerge from deliberative

processes are in some sense superior to those arrived at

via more traditional methods.19 We will return to these

questions in a moment.

Finally in this section, it is worth mentioning the

entirely different means of identifying deliberation

proposed by Nanz and Steffek.20 They argue that in the

particular case of international governance institutions,

in which any kind of mass participation is impossible for 19 It should be emphasised that the authors of the DQI make no claim

that it can be used as anything other than a means to identify

authentically deliberative processes and institutions. They

themselves argue that the really interesting questions arise only

once we have a reliable means of identification in hand and that the

point of the DQI is to provide that and nothing more. See Bächtiger

et al., ‘The Deliberative Dimensions of Legislatures’, p.226.18

pragmatic reasons, we can nevertheless evaluate the

deliberative quality of decision-making processes by asking

if ‘there is a warranted presumption that public opinion is

formed on the basis of adequate information and relevant

reasons, and that those whose interests are involved have

an equal and effective opportunity to make their own

interests (and their reasons for them) known’.21 Nanz and

Steffek’s operationalisation of this question involves

assessing the degree to which civil society organisations

(CSOs) are able to access and influence the policy- and

decision-making procedures of international organisations.

They propose four criteria: the access granted to CSOs (no

access, observer status or the right to speak in meetings

and submit documentation); the degree of transparency and

access to information (no access to documentation, access

to background documentation or full access to background

and policy documents); the responsiveness of international

organisations to stakeholder concerns (CSO concerns and

positions not discussed at all, state actors justify

proposals with reference to concerns voiced by CSOs or CSO

concerns become part of the agenda); and finally the degree

19

to which policy processes are actively inclusive (whether

or not institutional arrangements are made to ensure the

inclusion of all relevant CSOs).

The feasibility-capacity stream

Serious doubts have been expressed about the capacity of

individuals to deliberate in a way that meets the criteria

established in normative theory. As Ryfe puts it, we need

to ask whether deliberative democracy can ‘work at the

fundamental level of human reasoning’.22 There are at least

three aspects to this question: psychological, sociological

and (for want of a better word) educational.

Rosenberg, drawing both on his own research and a review of

other relevant psychology literature, argues that ‘only a

20 Patrizia Nanz & Jens Steffek, ‘Assessing the Democratic Quality of

Deliberation in International Governance: Criteria and Research

Strategies’, Acta Politica, 40(3) (2005), 368-383.

21 Nanz & Steffek, ‘Assessing the Democratic Quality of Deliberation

in International Governance: Criteria and Research Strategies’, p.

370.

22 David M. Ryfe, ‘Does deliberative democracy work? A state of the

field’, Annual Review of Political Science, 8 (2005), 49-71, p.50.20

small minority of individuals demonstrate deliberative

rationality, that is the requisite capacity to reflect on

their preferences and organize them with regard to higher

order goals or overarching life-plans... Similarly only a

small minority of people demonstrates deliberative

reasonableness and thus takes the perspective of another

and makes arguments that are persuasive in his terms’.23 A

number of other authors argue that opinion or preference

formation is only infrequently a matter of systematic

reasoning, more often involving a rather less cognitively

demanding process of responding to ‘heuristic cues’24 or

assessing new information for its compatibility with

existing beliefs and values or conceptual ‘frames’.25

Individuals may also change or adopt new opinions and

preferences as a result of ‘groupthink’, a process in which

‘likeminded or cohesive individuals either mutually

23 Shawn Rosenberg, ‘The Empirical Study of Deliberative Democracy:

Setting a Research Agenda’, Acta Politica, 40(2) (2005), 212-224, p.

221.

24 Ryfe, ‘Does deliberative democracy work ? A state of the field’ ;

Hans-Peter Kriesi, ‘Argument-Based Strategies in Direct-Democratic

Votes: The Swiss Experience’ Acta Politica, 40(3) (2005), 299-316.21

reinforce their existing perspectives… or engage in strong

consensus seeking behaviour so that it overwhelms any

potentially dissenting issues’.26

Scepticism about deliberation has also arisen from work in

sociology. Three particular problems have been identified

that, a priori, seem likely to have an impact on the

possibility of deliberation. The first is that the capacity

to deliberate and the ability to have ideas and opinions

taken seriously is unevenly distributed among different

members of society. For example, Sanders argues that not

only do the education and cultural capital associated with

the ability to argue a case convincingly vary with income,

25 Shane J. Ralston, ‘Intelligently Designing Deliberative Health Care

Forums: Dewey's Metaphysics, Cognitive Science and a Brazilian

Example’, Review of Policy Research, 25(6) (2008), 619-630.

26 Simon Niemeyer & John Dryzek, ‘Intersubjective rationality: Using

interpersonal consistency as a measure of deliberative quality’,

paper presented at the European Consortium for Political Research

35th Joint Sessions of Workshops, Helsinki, 2007, p. 17. Available

at:

http://deliberativedemocracy.anu.edu.au/documents/ECPSRHelsinkiPaper

_NiemeyerandDryzek2007.pdf [Accessed May 12, 2009].22

but inequalities are so deeply ingrained in our societies

that the simple fact of being, say, a black woman means

that your opinion is likely to carry less weight in

discussion.27 Hence, the argument goes, rather than

permitting participation on an equal basis regardless of

sex, race, class and so forth, deliberation may actually

tend to reinforce existing structures of power and

inequality. A second problem, closely related to the first,

is that deliberation is itself a culturally loaded

procedure that privileges calm, dispassionate modes of

discourse over overtly emotive but (arguably) equally valid

forms of expression like storytelling or testimony.

Participants from social backgrounds or cultures

characterized by these less ‘rational’ modes of expression

are thus prevented from making their voices heard.

Finally, the deliberative emphasis on consensus and

compromise, the search for what unites rather than what

divides, puts at risk the expression of difference and

particularity. As Sanders puts it, ‘In settings where there

27 Lynn M. Sanders, ‘Against Deliberation’, Political Theory, 25(3) (1997),

347-376.23

are gross inequalities in power and status, calling for

compromise may be perilously close to suppressing the

challenging perspectives of marginalised groups’.28

If these psychological and sociological critiques are well-

founded – and the question is clearly an empirical rather

than a theoretical one – then the basic conditions for

deliberation cannot be met. Even where an apparently

consensual decision emerges from discussion, it cannot be

assumed to have the privileged epistemic and normative

status that in principle accrues to the outcomes of

deliberative processes. The principal difference between

the psychologically- and sociologically-grounded critiques,

other than the fact that there is rather more concrete

evidence for the former, is that the psychologists are more

optimistic about the possibility of correcting the problem.

Although Neblo has suggested that there is ‘under-

appreciated room for mutual accommodation among

deliberative theorists on questions of emotion, rhetorica

and alternate communication forms’,29 most of the

sociological critics of deliberation seem unwilling to 28 Sanders, ‘Against Deliberation’, p. 362.

24

accept that any conceivable intervention would correct the

difficulties they identify.30 On the other hand, Delli

Carpini and his colleagues, argue that although the

research evidence shows that successful deliberation is

highly context-dependent and ‘rife with opportunities for

going awry’,31 it also provides a great deal of indirect

support for the democratic potential of deliberation.

Rosenberg takes the research evidence as a challenge to be

met rather than an immovable obstacle:

if deliberations were designed with the limitations of

the participants in mind, two goals might be

accomplished. First, more reasoned and just processes

and outcomes might be obtained. Second, the

deliberation might provide a context for the further

development of the participants’ existing deliberative

capacities... In either case, the consideration of the

nature and design of deliberative institutions

necessarily shifts away from the more typical focus on

establishing conditions and processes that free

individuals to do what they already can. Instead

25

attention turns to establishing conditions and

processes that guide individuals so that they can

achieve a potential that they possess but have not

realized.32

Work carried out by researchers at the Australian National

University’s deliberative democracy research group provides

further important evidence both that deliberation is a

distinctive socio-cognitive process and that it has the

29 Michael A. Neblo, ‘Family Disputes: Diversity in Defining and

Measuring Deliberation’, Swiss Political Science Review, 13, 527-557 (2007),

p. 535.

30 See for example Chantal Mouffe, ‘Deliberative Democracy or

Agonistic Pluralism?’, Social Research, 66(3), 745-758; Sanders,

‘Against Deliberation’; Elaine Stratford, Denbeigh Armstrong &

Martina Jaskolski, ‘Relational Spaces and the Geopolitics of

Community Participation in Two Tasmanian Local Governments: A Case

for Agonistic Pluralism?’, Transactions of the Institute of British Geographers,

28(4) (2003), 461-472.

31 Michael X. Delli Carpini, Fay L. Cook & Lawrence R. Jacobs, ‘Public

deliberation, discursive participation and citizen engagement: A

Review of the Empirical Literature’, Annual Review of Political Science, 7(1)

(2004), 315-344, p. 328.26

effects predicted by normative theory for the reasons that

these effects are expected.33 As we saw above, one of the

two most basic conceptual criteria for deliberation is

reason-giving. Both the epistemic value and normative

legitimacy of decisions reached via deliberative methods

arise from consensus on the underlying reasons for taking

some collective action. On this basis, the ANU researchers

argue that an increase in what they call ‘intersubjective

consistency’ can be used as a measure of the effectiveness

of deliberation. Intersubjective consistency occurs when a

group of individuals not only have common preferences, but

also share the beliefs and values (subjectivity) that give

rise to those preferences. While a convergence of

preferences alone may be the result of non-deliberative

social processes, where the subjective reasons underlying

preference formation also converge as a result of

discussion and debate we have strong evidence that

deliberation has occurred. The ANU researchers have been

able to show that intersubjective consistency does indeed

32 Rosenberg, ‘The Empirical Study of Deliberative Democracy: Setting

a Research Agenda’, p. 222.27

increase as a result of deliberation.

The third aspect of the feasibility-capacity research

stream is concerned not with the inherent intellectual or

psychological capacities of participants in deliberation,

but with their level of education and their knowledge of

the issues under discussion. A potential problem that is

frequently raised is that the technical complexity of

certain policy issues is such that most participants will

be unable to grasp them sufficiently well for meaningful

deliberation to be possible. For example, in a discussion

of the ‘transition’ economies of central and eastern

Europe, Przeworski argues that market economic reforms ‘are

based on a model of economic efficiency that is highly

technical. They involve choices that are not easy to

explain to the general public and that do not always make

sense to popular opinion’.34 However, the evidence that non-

expert participants in deliberation are capable of making

sense of complex technical issues is very strong.35

Esterling and his colleagues also found that participants

in deliberation use the process precisely to become

28

informed about policy issues,36 while Ryfe suggests that

participants who are knowledgeable about a particular issue

may actually be less able to deliberate effectively on that

issue than those who are not. Ryfe argues that expert

participants tend to be ‘more rigid in their thinking, less

tolerant of others’ views and, perhaps most importantly,

more adept at rationalizing their own’.37

The institutional-environmental stream

The impact of four main types of contingent factor external

to deliberative processes has been considered in this

stream in the literature: institutional characteristics

(whether the institutions in question are specifically

designed to promote deliberation or not); the features of

the social and political environment; the initial political

and value commitments of participants; and the nature of

the issues under discussion.

There is little consensus on what institutional features 33 Niemeyer et al. ‘Achieving Success in Large Scale Deliberation: An

Analysis of the Fremantle Bridge Community Engagement Process’;

Niemeyer & Dryzek, ‘Intersubjective rationality: Using interpersonal

consistency as a measure of deliberative quality.’29

make for an effective deliberative forum, the range of

possible designs being extremely wide. As Fung puts it,

‘The menu of institutional alternatives is far richer than

the dichotomy between representative and participatory

democracy supposes, and most of the items on that menu

remain empirically and normatively unexplored’.38 A detailed

assessment of the pros and cons of different types of forum

is beyond the scope of this review, but we can at least

indicate the principal dimensions of the problem. In the

literature, discussion focuses on the aim or purpose of the

forum, the integration of the forum into the larger

political process, the selection of participants (including

professional advocates or other stakeholders), the

organisation of participants within the deliberative

process and the role played by experts.

An issue raised by several authors is the ultimate aim or

purpose of the deliberative forum. For example, Setälä et

al distinguish between processes that require participants

to come up with some kind of collective conclusion and

those that allow participants to express an individual

30

opinion at the end of the process by voting or responding

to an attitude survey or poll.39 This is related to the

issue of what Grogan and Gusmano call the ‘purpose and

style’ of deliberation.40 Citing the work of Button &

Mattson, they outline four possible models: educative,

intended to provide participants with information and

knowledge; consensual, aimed at finding agreement on an

issue, value or plan for the future; activist/instrumental,

leading to political or legislative action; and conflictual,

emphasising the provision of the widest possible space for

the expression of different points of view.41

Related to the question of what deliberative forums are for

is the issue of the connection between the decision or

policy of a forum and the larger political process. Goodin

and Dryzek list eight different ways in which deliberative

forums can be integrated into politics, from actually

making policy to simply informing public debates.42 Dryzek

and Tucker suggest that the output of deliberative forums

can be deployed in integrative or managerial ways or as

advocacy.43 In the first case, the output of deliberation ‘is

31

integrated into the established policy making structure,

and is designed to help integrate informed public opinion

and key actors into that structure’.44 In the second case,

the deliberative forum is ‘used as a tool through which key

policy makers … produce an alternative representation of

34 Cited in Dryzek, ‘Democratization as deliberative capacity building’,

p.13.

35 See for example John Dryzek & Aviezer Tucker, ‘Deliberative

innovation to different effect: Consensus conferences in Denmark,

France and the United States’, Australian National University, 2005.

Available at:

http://deliberativedemocracy.anu.edu.au/documents/DryzekandTucker200

5.pdf [Accessed May 10, 2009]; Kaspar M. Hansen, Vibeke N. Andersen,

‘Deliberative democracy and the deliberative poll on the euro’,

Scandinavian Political Studies, 27(3) (2004); Giorgos Kallis, Dionyssia

Hatzilacou, Alexandra Mexa, Harry Coccossis & Eleni Svoronou,

‘Beyond the manual: Practicing deliberative visioning in a Greek

island’, Ecological Economics, 68 (2009), 979-989; Carolyn M. Hendriks,

John Dryzek & Christian Hunold, ‘Turning up the heat: partisanship

in deliberative innovation, Political Studies, 55 (2007), 362-383.

36 Kevin M. Esterling, Michael A. Neblo, David M. Lazer, ‘Means,

Motive, & Opportunity in Becoming Informed About Politics: A

Deliberative Field Experiment’, paper presented at the annual 32

informed public opinion to contrast with the uninformed

skepticism of the mass public’ on the issue in question.45

In the third case, the results of deliberation are

mobilized by civil society actors in lobbying and advocacy

work.

While participation in ‘town meeting’ style deliberative

forums46 tends to be entirely open or self-selecting, most

other types of forum adopt a representativity rule for the

selection of participants. Deliberative polls, for example,

aim to attract a set of participants that, while otherwise

randomly selected, is as statistically representative of

the relevant population as possible.47 Other types of forum

operate on the basis of descriptive representation or

‘social mirroring’ as Hendriks calls it.48 As Goodin and

Dryzek put it, what this means is that ‘the diversity of

meeting of the Americal Political Science Association, Chicago,

2007. Available at:

http://www.hks.harvard.edu/netgov/files/png_workingpaper_series/PNG0

7-006.pdf [Accessed June 8, 2009].

37 Ryfe, ‘Does deliberative democracy work ? A state of the field’, p.

5.33

social characteristics and plurality of initial points of

view in the larger society are substantially present in the

deliberating mini-public. Social characteristics and

viewpoints need not be present in the same proportions as

in the larger population’.49 Descriptive representativity

arguably includes those situations in which certain social

groups are represented by professional advocates who, while

not elected, have some warranted claim to be able to speak

on behalf of those groups. Finally, there is electoral

representativity which, obviously enough, involves

participants who are elected by and are accountable to

certain sections of society. Although no conclusions seem

to have been drawn about whether statistical or descriptive

representativity makes for better deliberation, there is

some agreement that whether or not participants are

‘partisan’ is an important variable. The distinction

between partisan and non-partisan participants turns

essentially on whether or not they have relatively open

38 Archon Fung, ‘Democratic Theory and Political Science: A Pragmatic

Method of Constructive Engagement’, American Political Science Review, 101(3)

(2007), p. 445.34

preferences on the issue under discussion. As to the

question of whether partisanship is a good or a bad thing,

Hendriks et al found that ‘[w]hile the partisan

participants [in their study] accepted that deliberation

requires willingness to adjust preferences, they failed to

translate this into action and held on tightly to their

positions. Our case suggests that partisans might be

morally committed to the idea of deliberation, but they

struggle to put it into practice’.50 On the other hand,

Grogan and Gusmano argue that while it is crucial to

involve an appropriate range of descriptively 39 Maija Setälä, Kimmo Grönlund & Kaisa Herne Citizen deliberation on

nuclear power: A Comparison of two decision-making methods.

University of Berne Center for Interdisciplinary Deliberation

Studies, 2004. Available at:

http://www.bids.unibe.ch/unibe/rechtswissenschaft/oefre/bids/content

/e3409/e3822/e3824/linkliste3828/Setl:Grnlund:Herne.pdf [Accessed

June 7, 2009].

40 Colleen M. Grogan & Michael K. Gusmano, ‘Deliberative democracy in

theory and practice: Connecticut's medicaid managed care council’

State Politics and Policy Quarterly, 5(2) (2005), 126-146.

41 Grogan & Gusmano, ‘Deliberative democracy in theory and practice:

Connecticut's medicaid managed care council’, p. 132.35

representative but otherwise non-partisan participants,

this should not be seen as an alternative to the presence

of professional advocates for the interested social groups

because these professional partisans are more inclined to

challenge the views of experts.51

The organisation of participants within the deliberative

process – their allocation to different working groups, the

relationship between these groups and any plenary

discussion sessions and so forth – seems in most cases to

be of relatively little interest. However, two cases are

worth mentioning. Davies and Burgess report on a

deliberative forum dealing with the allocation of organs

for transplantation in which participants were segregated

by both sex and socio-economic status.52 The authors found

significant differences in the discursive practices of the

42 Robert E. Goodin & John S. Dryzek, ‘Deliberative Impacts: The

Macro-Political Uptake of Mini-Publics’, Politics & Society, 34(2) (2006),

219-244.

43 Dryzek & Tucker, Deliberative innovation to different effect:

Consensus conferences in Denmark, France and the United States.

45 Dryzek & Tucker, op. cit., p. 22.36

male and female groups, in particular with respect to their

interaction with the healthcare professionals advising the

forum. In their study of educational segregation,

Mendelberg and Oleske report on a series of open-

participation town meetings which were spontaneously

segregated by race.53 This segregation proved to be a

serious obstacle to deliberative interaction as the

arguments put forward by participants remained within the

bounds of types of discourse which, while largely held to

be valid within each racial group, were not so recognized

outside them or, worse, were interpreted as coded forms of

racism.

There is rather less research on the institutional and

procedural characteristics that are associated with

deliberative forms of interaction in ‘normal’ democratic

political contexts – as opposed to forums specifically

designed to permit deliberation. Nevertheless, the DQI and

similar approaches have been able to shed some light on the

relationship between the nature of discussion and debate

that takes place within legislative bodies, different types

37

of political institutions and different aspects of the

political environment. Bächtiger and his colleagues found

that five factors were of relevance.54 Presidential systems

of government, in which the executive and legislative

functions are strictly separated, a political culture that

emphasizes consensus over competition, debate in second

chambers, non-public debate and low issue polarisation were

all found be associated with higher quality discourse.

Falling somewhere between the DQI and the Australian

National University’s intersubjective consistency approach

we find the work of Karlsson on the European Convention,

the representative group that prepared the text of the ill-

fated European constitution.55 On the basis of interviews

with participants in the Convention, Karlsson attempts to

determine to what extent participants went into the process

with open minds about what the outcomes might be; the

degree to which they actually did change their views in the

course of the process; and the degree to which this change

was due to a rational engagement with arguments made by

other participants rather than a consequence of, for

38

example, threats or political expediency. He found that the

participants’ widely-shared view that the process was more

deliberative than negotiated was consistent with their

accounts of the development of their opinions and their

narratives of how the Convention had unfolded.

44 Dryzek & Tucker, op. cit., p. 15.

46 Mansbridge, Beyond Adversary Democracy; Mandelberg & Oleske, ‘Race &

Public Deliberation’.

47 James S. Fishkin & Robert C. Luskin, ‘Experimenting with a

Democratic Ideal: Deliberative Polling and Public Opinion’, Acta

Politica, 40(3) (2005), 284-298; Robert C. Luskin, James S. Fishkin &

Roger Jowell, ‘Considered opinions: Deliberative polling in

Britain’, British Journal of Political Science, 32 (2002), 455.

48 Carolyn M. Hendriks, ‘On inclusion and network governance: the

democratic disconnect of Dutch energy transitions’, Public Administration,

86(4) (2008), 1009-1031.

49 Goodin & Dryzek, ‘Deliberative Impacts : The Macro-Political Uptake

of Mini-Publics’, p. 221.

50 Hendriks et al, ‘Turning up the heat: partisanship in deliberative

innovation’, p. 370.

51 Grogan & Gusmano, ‘Deliberative democracy in theory and practice:

Connecticut's medicaid managed care council’.39

The issue of open-mindedness and the possibility of

preference change has also been examined by Hendriks and

her colleagues in their work on the effect of partisanship

on deliberation.56 ‘Partisans’ are those who possess firm

and established opinions on an issue, or who are active

advocates for a particular group or point of view. The

researchers studied existing forums for discussion, debate

and decision-making, dividing them into those in which

partisan participants were directly involved in

deliberation and those in which they were merely invited to

present and defend their position before panels of ‘lay’

citizens. The different types of forum were compared on the

basis of their deliberative capacity, the legitimacy of

their procedures and decisions in the view of key

stakeholders and the media, and the political impact of

their findings. Arguably the most important finding of the

52 Gail Davies & Jacquelin Burgess, ‘Challenging the ‘view from

nowhere’: citizen reflections on specialist expertise in a

deliberative process’, Health & Place, 10 (2004), 349-361.

53 Mendelberg & Oleske, ‘Race & Public Deliberation’.

54 Bächtiger et al, ‘The deliberative dimensions of legislatures’.40

research was that the partisan forums had lower

deliberative capacity than the non-partisan forums: ‘While

the partisan participants accepted that deliberation

requires willingness to adjust preferences, they failed to

translate this into action and held on tightly to their

positions. Our case suggests that partisans might be

morally committed to the idea of deliberation, but they

struggle to put it into practice’.57 The partisan forums

were also perceived as having lower legitimacy, although

this may have been due to the means of measuring legitimacy

used in the research. As the authors put it, ‘Legitimacy in

the partisan case turned largely on whether the

deliberations produced the preferred outcome for the

organizations with an interest in the case’.58 On the other

hand, the authors report that these same organisations

criticized the non-partisan forums on the grounds that the

participants did not properly appreciate the ‘science’ of

the issues, nor what it was like to be ‘affected’.

Finally in this section we should mention the issue

dimension, which is to say the type of policy issue or 55 Karlsson, ‘Deliberation at the European Convention’.

41

decision under discussion. We have already considered the

question of the scientific or technical complexity of the

subject matter of deliberation. Beyond this, the literature

on the topic is not extensive. Few if any researchers have

been able to go beyond suggesting that non-contentious

issues lend themselves rather more easily to high quality

deliberation than, say, stem cell research, and that the

more concrete the issue – the closer the deliberative forum

is to having to make an actual final decision – the more

likely it is that participants will adopt rather entrenched

and inflexible positions. For example, Papdopoulos, citing

work carried out by Klaus Eder, reports that ‘Research on

deliberative institutions of environmental policy in

Germany ... shows that their contribution to consensus-

building is higher when concrete decisions are not at

stake, and confirms that the initial level of polarisation

should not be too high’.59

56 Hendriks et al, ‘Turning up the heat: partisanship in deliberative

innovation’.

57 Hendriks et al, op.cit., p. 370.

58 Hendriks et al, op. cit., p374.42

Methodological approaches

In this final section of our review we will give a brief

overview of the methodological approaches adopted by

empirical deliberation researchers. Once again we can

identify a number of different – but non-exclusive –

streams in the literature. First of all there are those

approaches based on the direct analysis of discursive

practice ‘as it happened’. Steiner et al’s DQI is perhaps

the best example in this category, although we have also

mentioned the work of Holzinger and Conover and Searing.60

In all three of these cases, the researchers apply formal

content analysis to the transcripts of discussions, coding

different types of speech or the attitudes manifested by

participants in order to carry out some kind of statistical

analysis. Other researchers also make use of transcripts of

discussions but without using formal analytic techniques.61

A second stream in the literature involves the discussion

59 Yannis Papadopoulos, ‘Towards some research questions on the merits

and limits of deliberative policy-making’, paper presented at the

conference Empirical Approaches to Deliberative Politics, European University

Institute, Florence, 2004, p. 9.43

and analysis of participants’ post-deliberation reflections

on the process and outcomes of deliberation. This approach,

generally involving semi-structured interviews, is clearly

the most widely used, whether as a principal research

method or in support of other more formal techniques.

Karlsson’s work on the European Convention, for example, is

based exclusively on interviews, while Niemeyer and his

colleagues combine post-discussion interviews with

participants with both informal content analysis of

discussion transcripts and survey techniques.62

Work involving survey methods represents a third

substantial stream in the literature. Deliberative polling

is the best known and best established of the survey-based

empirical approaches, functioning essentially via a

60 Steiner et al, Deliberative Politics in Action ; Holzinger, ‘Context or

conflict types’ ; Conover & Searing, ‘Studying ‘Everyday Political

Talk’ in the Deliberative System’.

61 See for example Davies & Burgess, ‘Challenging the ‘view from

nowhere’’; Kallis et al, ‘Beyond the manual’.

62 Karlsson, ‘Deliberation at the European Convention’; Niemeyer et al,

‘Achieving success in large-scale deliberation’.44

relatively straightforward comparison of participants’

expressed preferences before and after deliberation. More

recently, the Australian National University research group

has made a good case for the use of Q-methodology.63 This is

a type of factor analysis performed on subjects’ rankings

of their degree of agreement or disagreement with 30 to 60

statements about an issue. The Q factor analysis provides a

picture of each subject’s subjective beliefs at a

particular point in time and so permits a comparison not

just of preferences before and after deliberation, but the

reasons for which those preferences are held.

Finally, we have those methodological approaches that focus

on the analysis of institutional characteristics or the

structure of deliberative processes and the effect of

contingent environmental factors. This type of approach is

most frequently found in conjunction with other techniques

aiming to evaluate the quality of discursive practices or

the outcomes of deliberation, although the work of Nanz and

Steffek is an unusual example of an attempt to measure the

63 Niemeyer & Dryzek, ‘Intersubjective Rationality’; Niemeyer et al, op.

cit.45

democratic quality of institutions as an end in itself.64 In

most of these cases, the analysis involves the

categorisation of institutional or environmental factors

into two or more types that might be expected to have a

different effect on discursive practice. Hence we have, for

example, Dryzek’s categorisation of political systems

according to their openness to influence from organized

activist groups or Hendriks’s division of deliberative

forums into partisan and non-partisan.65

Conclusions

In this review we have described the principal theoretical

and technical issues involved in empirical research on

deliberative democracy and decision-making. We have seen

that, insofar as it addresses the empirical conditions for

deliberation, existing research can be divided into three

streams which we have called the behavioural-procedural,

feasibility-capacity and institutional-environmental 64 Nanz & Steffek, ‘Assessing the Democratic Quality of Deliberation in

International Governance’.

65 Dryzek & Tucker, ‘Deliberative Innovation to Different Effect’;

Hendriks et al., ‘Turning up the Heat’.46

streams. Notable by its almost complete absence, however,

is any work that aims to evaluate the technical

effectiveness of deliberative policy- and decision-making

relative to other types of process, democratic or

otherwise. Although a number of authors pose the question,66

it is only to note that it has not (yet) been answered.

This is all the more surprising bearing in mind that one of

the most frequent claims in the normative literature is

that deliberation has a strong epistemic or truth-tracking

potential. In principle, then, it ought to lead to the best

possible policy outcomes, although as Habermas puts it,

‘Whether deliberation does indeed introduce an epistemic

dimension into political will-formation and decision-making

is, of course, an empirical question”.67

47

66 See, for example, Patricia Fitzpatrick, A. John Sinclair & Bruce

Mitchell, ‘Environmental impact assessment under the Mackenzie

Valley Resource Management Act: Deliberative democracy in Canada's

North?’ Environmental Management, 42(1) (2008), 1-18, p.16; David M.

Ryfe, ‘The practice of deliberative democracy: a study of 16

deliberative organizations’, Political Communication, 19 (2002), 359-377,

p.370.

67 Jürgen Habermas, ‘Political Communication in Media Society: Does

Democracy Still Enjoy an Epistemic Dimension? The Impact of

Normative Theory on Empirical Research’ Communication Theory, 16 (206),

411-426, p.413.48