Post on 22-Jan-2023
Empirical Research on Deliberative Democracy: A
Review of the Literature
Abstract
This paper reviews the recent literature on empirical
research into deliberative decision-making. Following
Thompson, it distinguishes between the conceptual criteria,
evaluative standards and empirical conditions associated
with good or effective deliberation. It finds that the
question of empirical conditions has been approached from
three distinct directions. In one stream in the literature,
which it calls the behavioural-procedural stream, work has
focused on producing an empirical definition of
deliberation which permits an assessment of the degree to
which deliberation is occurring. A second stream in the
literature, the feasibility-capacity stream, has involved seeking
evidence to support (or undermine) the claims of normative
theory about deliberation as a social process. A third
stream, the institutional-environmental or boundary conditions
stream, involves considering the effect on discursive
practice of a series of contingent factors external to the
process of deliberation itself. Notable by its almost
complete absence, however, is any work that aims to
evaluate the technical effectiveness of deliberative
policy- and decision-making relative to other types of
process, democratic or otherwise.
Introduction
In this paper we review a selection of the recent
literature reporting the results of empirical research into
deliberative decision-making. This current of research
appears to be a response to the increasing incidence of and
interest in participatory approaches to governance and
policy decision-making. These approaches are in turn a
response to the perception that existing modes of social
and political governance lack legitimacy. Hartz-Karp and
Briand, for example, argue that in advanced societies,
the public has unquestioningly handed over much of the
necessary decision-making to ‘experts’, to whom
implicitly we have assigned the ability to determine
what counts as knowledge and what does not. Thus
2
policy-makers too readily accept their own views as
sound, but treat the views of ordinary citizens – even
in regard to matters properly within their realm of
‘expertise’, such as values and priorities – as mere
‘preference’ and ‘opinion’’1
In a similar vein, Wagenaar suggests that
the increasing technical complexity of societal
sectors, in combination with a complex cross-current of
administrative ideologies that favor expertness,
managerial efficacy, and allocative efficiency, has
resulted in rather a high-handed, technocratic style of
policy making in which administrators and external
experts claim the right to manage entire societal
sectors and in which the citizens who are at the
receiving end of these policies are consequently
disenfranchised from the governance of their own
environment.2
Political participation can take a dizzying array of
different forms, but Huitema and his colleagues have
3
usefully parsed these into those arising from a pluralist
discourse and those that emerge from the discourse of
deliberation.3 The emphasis in the former case is on
participation as a means of guaranteeing the effective
representation of individuals and groups in a political
process based on balancing competing values and interests.
In the latter case, by contrast, the emphasis is on the
identification and articulation of a single authentic
collective interest – Rousseau’s ‘general will’ as opposed
to the mere ‘will of all’ – and the assembly of a reasoned
argument for the pursuit of a particular course of social
or collective action in the light of that shared interest.4
According to Baccaro & Papadakis, the principal claim of
the advocates of deliberative participation is that it
‘produces not just a richer texture of democracy but also
more effective public policy’.5 One might add that,
particularly for those authors who adopt broadly 1 Janette Hartz-Karp & Michael K. Briand, ‘Institutionalizing
deliberative democracy’, Journal of Public Affairs, 9(2) (2009), p.135-6.
2 Hendrik Wagenaar, ‘Governance, Complexity, and Democratic
Participation’ American Review of Public Administration, 37(1) (2007) 17-50,
p.22.4
Habermasian theoretical approaches, the increased
effectiveness of policy outcomes is held to be a direct
result of the democratic nature of the decision-making
process.
While there is some doubt about whether a sharp distinction
between pluralist and deliberative forms of participation
can be maintained in practice, in this review we will
nevertheless confine ourselves to considering those forms
of democratic participation that, at least as a regulatory
ideal, aim to seek consensus not just on action but on the
technical and normative reasons for action. The books and
papers referenced in the review were identified via a trawl
of the websites of the main academic research centres and 3 David Huitema, Marleen Kerkhof & Udo Pesch, ‘The nature of the
beast: are citizens' juries deliberative or pluralist?’ Policy Sciences,
40 (2007), 287-311.
4 Mansbridge uses the terms ‘adversary’ and ‘unitary’ democracy in
drawing essentially the same distinction. See Jane Mansbridge, Beyond
Adversary Democracy, (Chicago: University Of Chicago Press, 1980).
5 Lucio Baccaro & Konstantinos Papadakis, ‘The downside of
participatory-deliberative public administration’, Socio - Economic
Review, 7(2) (2009), p. 246.5
principal figures known to be involved in work on
deliberative democracy, together with a closely-focused
search of electronic journals.6 The search requested
articles with the words ‘deliberation’ or ‘deliberative’
together with the word ‘empirical’ in the title or
abstract. In the end a total of around 75 journal articles
and working papers were reviewed. The relatively narrow
overall search strategy means that most work on non-
deliberative forms of participatory governance and
democracy is likely to have been excluded.
What is Deliberative Democracy?
Thompson suggests that there are three elements in the
analysis of political deliberation that need to be
distinguished: conceptual criteria, evaluative standards,
and empirical conditions. ‘Conceptual criteria stipulate
what is necessary for a practice to count as deliberation.
Evaluative standards specify what counts as good (or
better) deliberation. Empirical conditions indicate what is
necessary for producing good deliberation (or less
6 The literature search was carried out in July 2009.6
strongly, what may contribute to producing good
deliberation)’.7 We will use Thompson’s analytical model to
structure the discussion that follows.
Conceptual Criteria
While, as Rosenberg points out,8 work in the anglo-american
political science tradition, particularly that of John
Rawls, has been of great significance in the development of
work on deliberative democracy, the writing of Jürgen
Habermas seems to have been the most important catalyst for
the increasing interest in deliberation apparent from the
mid-1990s. This is reflected in the relatively narrow range
of conceptual definitions of deliberative democracy that
feature in the literature. As Schneiderhan and Khan point
out,9 almost without exception these definitions focus on
reason-giving and inclusiveness, thereby taking up the essentials
of Habermas’s concept of communicative action and his
discourse principle. Thus normative theories of
deliberative democracy emphasize communication about
7 Dennis F. Thompson, ‘Deliberative Democratic Theory and Empirical
Political Science’ Annual Review of Political Science, 11(1) (2008), 497-520,
p.501.7
preferences rather than the aggregation of fixed preferences
via individual choice or voting.10 Normative theory also
proposes that ‘democratic legitimacy resides in the right,
ability and opportunity of those subject to a collective
decision to participate in deliberation about the content
of that decision’.11 Although there no suggestion in the
literature that inclusiveness is the less important of the
two principles, it is certainly less uniquely
characteristic of deliberation than reason-giving. As
Thompson puts it, ‘the most important distinguishing
characteristic of deliberation, is mutual justification –
presenting and responding to reasons intended to justify a
political decision’.12
Evaluative standards
In the literature the primary evaluative standards applied
to reason-giving are:
• that reasons are couched in terms of the common good,
including the good that arises from permitting
individuals or groups to pursue their particular
8
interests in certain circumstances;
• that reasons are used sincerely and/or truthfully; and
• that there is a logical relation between a given reason
and the preference it supports.13
8 Shawn Rosenberg, ‘An Introduction: Theoretical Perspectives and
Empirical Research on Deliberative Democracy’ in Shawn Rosenberg,
ed., Deliberation, Participation and Democracy: Can the People Govern? (London:
Palgrave Macmillan, 2007).
9 Erik Schneiderhan, & Shamus Khan, ‘Reasons and Inclusion: The
Foundation of Deliberation’, Sociological Theory, 26(1) (2008), 1-24.
10 André Bächtiger, Marco Steenbergen & Axel Tschentscher, ‘Developing
deliberative democracy: A research report and a research agenda’,
University of Berne Institute for Interdisciplinary Deliberation
Studies (2008) Available at:
http://www.bids.unibe.ch/unibe/rechtswissenschaft/oefre/bids/content
/e3409/e3822/e3824/linkliste3826/
Bchtiger:Steenbergen:Tschentscher.pdf [Accessed May 12, 2009].
11 John Dryzek, ‘Democratization as deliberative capacity building’,
University of Berne Institute for Interdisciplinary Deliberation
Studies (2008), p 3. Available at:
http://www.bids.unibe.ch/unibe/rechtswissenschaft/oefre/bids/content
/e3409/e3822/e3824/linkliste3831/Dryzek.pdf [Accessed May 12, 2009].9
As applied to the concept of inclusiveness, we find
standards that evaluate not so much whether the relevant
subjects or their representatives are formally involved in
the deliberative process, but the degree to which
participants in deliberation are genuinely able to
contribute to the discussion and to have their voices
heard. Thus we have the standards of:
• participation in the sense of the availability of
substantive opportunities to contribute to the
discussion;
• respect for the arguments and the perspective of other
12 Thompson, ‘Deliberative Democratic Theory and Empirical Political
Science’, p.504.
13 See for example Jürg Steiner, André Bächtiger, Markus Spörndli &
Marco Steenbergen. Deliberative Politics in Action: Analysing Parliamentary Discourse
(Cambridge: Cambridge University Press, 2004); Wagenaar,
‘Governance, Complexity, and Democratic Participation’; Robert E.
Goodin, ‘Sequencing Deliberative Moments’ Acta Politica, 40(2) (2005),
182-196. Dryzek, ‘Democratization as deliberative capacity
building’; Tali Mendelberg & John Oleske, ‘Race and Public
Deliberation’, Political Communication, 17 (2000), 169-191.10
participants; and
• openness to changing preferences in response to the force
of the better argument.14
In truth this last standard applies to both reason-giving
and inclusiveness. Reason-giving obviously loses its
effectiveness as a means of reaching agreement the more
unwilling participants are to be persuaded away from their
starting positions. It is also true that it is pointless to
include a wide range of participants in a deliberative
process if there is no possibility that they might change
their minds in response to the different perspectives 14 See for example André Bächtiger & Jürg Steiner, ‘Introduction’, Acta
Politica, 40(2) (2005), 153-168; Christer Karlsson, ‘Deliberation at
the European Convention: The Final Verdict’, European Law Journal, 14(5)
(2008), 604-619; Simon Niemeyer, S., Selen Ayirtman & Janette Hartz-
Karp, ‘Achieving Success in Large Scale Deliberation: An Analysis of
the Fremantle Bridge Community Engagement Process’, Australian
National University (2008), Available at:
http://deliberativedemocracy.anu.edu.au/Frembridge/FremBridgeRpt.pdf
[Accessed May 11, 2009]; Michael Neblo, ‘Thinking through Democracy:
Between the Theory and Practice of Deliberative Politics’, Acta
Politica, 40(2) (2005), 169-181.11
brought to the discussion by others.
Empirical conditions
The question of the empirical conditions associated with
good or effective deliberation has been approached from
three distinct directions. In one stream in the literature,
which we can call the behavioural-procedural stream, work has
focused on producing an empirical definition of
deliberation which permits an assessment of the degree to
which deliberation is occurring. The conditions specified
tend to be a fairly straightforward operationalisation of
the evaluative standards we have just discussed, and thus
focus on the attitudes and discursive behaviour of
participants as well as the logic and structure of the
arguments they propose. In this stream, the possibility of
deliberation as a distinct mode of discursive interaction
is not in question. Rather, specifying the empirical
conditions for deliberation is merely an intermediate step
towards answering more substantive questions about the
outcomes and effects of deliberation.
12
A second stream in the empirical research literature, the
feasibility-capacity stream, has involved seeking evidence to
support (or undermine) the claims of normative theory about
deliberation as a social process. Researchers contributing
to this stream do not assume that deliberation is always
(or, in some cases, ever) possible, and tend to be
concerned with the social psychology of discursive practice
and with power relations arising from differences of class,
sex and ethnicity. They are interested in whether
supposedly deliberative interactions have the claimed
effect – consensus, legitimacy and so on – as well as the
extent to which the effects of such interactions can be
traced to social and psychological processes that are
coherent with normative theory.
A third stream in the empirical literature, the institutional-
environmental or boundary conditions stream, is slightly
different in that it overlaps with both of the other
approaches and is potentially coherent with either. This
approach involves considering the effect on discursive
practice of a series of contingent factors external to the
13
process of deliberation itself.
The behavioural-procedural stream
The discursive behaviour of participants has been an
important focus of empirical research on deliberation that
aims to assess the ‘deliberativeness’ of existing political
institutions and processes. For example, Holzinger has
analysed the debate in the German parliament on the
contentious issue of stem cell research.15 Her work, based
on the analysis of transcripts of the sessions in question,
seeks to assess the relative importance of arguing and
bargaining in the debate. This latter distinction is based
on J L Austen and John Searle’s work on speech acts.
Arguing involves speech acts such as claiming,
establishing, assuming, asking, justifying, contradicting
and judging, whereas bargaining involves an entirely
different set of discursive practices like demanding,
offering, promising, threatening, accepting and rejecting.
What Holzinger’s analysis shows, however, is that this
15 Katharine Holzinger, ‘Context or Conflict Types: Which Determines
the Selection of Communication Mode’, Acta Politica, 40(2) (2005), 239-
254.14
distinction is not adequate to the identification of
deliberative behaviour. As she herself concedes, there was
a great deal of arguing and very little bargaining in the
debate, as would be expected from the nature of the issue
in question, but very little of this argument was
deliberative. Rather, non-dialogical rhetoric was the
dominant form of speech, no consensus was reached and the
issue was ultimately resolved by voting.
Conover and Searing come at the question of identifying
deliberation rather more directly in their analysis of
political talk.16 Analysing the transcripts of focus groups
and, less conventionally, the text of letters published in
British newspapers, the authors seek to assess the extent
to which political talk satisfies their version of the
empirical conditions for deliberation: reciprocity,
publicity, non-tyranny and equality. They conclude that in
general, this kind of talk falls well short of the
deliberative ideal: ‘Although citizens are committed to
16 Pamela J. Conover & Donald D. Searing, ‘Studying ‘Everyday
Political Talk’ in the Deliberative System’, Acta Politica, 40(3)
(2005), 269-283.15
reciprocity in principle, in practice they sometimes find
it difficult to respect their fellow citizens. Public
discussions occur infrequently, for most everyday talk is
relatively private in nature. Finally, while everyday talk
typically involves weak contestation, it also reflects
inequalities in society, particularly when political
discussion becomes more public’.17
Perhaps the best-known attempt to provide an empirical
‘test’ for deliberation is the work of Steiner and his
colleagues at the University of Berne.18 This work, which
aims to evaluate the quality of debate in national
legislatures with respect to the evaluative standards of
deliberation, revolves around the ‘discourse quality index’
or DQI. The DQI provides a measure of the deliberativeness
of the discursive behaviour of legislators along the four
17 Conover & Searing, ‘Studying ‘Everyday Political Talk’ in the
Deliberative System’, p. 278.
18 Steiner et al., Deliberative Politics in Action: Analysing Parliamentary Discourse;
André Bächtiger, Markus Spörndli, Marco Steenbergen & Jürg Steiner,
‘The Deliberative Dimensions of Legislatures’, Acta Politica, 40(2)
(2005), 225-238.16
principal dimensions of participation, justification
(divided into level and content of justification), respect
(divided into respect for other groups, respect for group
interests and demands and respect for counterarguments) and
constructive politics (consensus-building). Participation
reflects the opportunity afforded to legislators to speak
without interruption. Justification is a measure of the
sophistication with which arguments are grounded and of the
degree to which arguments are couched in terms of the
common good as opposed to group interests. Respect measures
the degree to which participants’ interventions reflect
negative or positive attitudes to opposing groups,
interests and arguments. Finally, constructive politics is
a measure of the extent to which legislators make efforts
to find universally acceptable solutions as opposed to
simply trying to insist that their particular view prevail.
The DQI framework is arguably the most methodologically
sophisticated approach to an operationalization of the
evaluative standards for deliberation, despite its
(deliberate) omission of any means of assessing the
17
sincerity or truthfulness of participants. However, even if
it were possible effectively to address this lacuna, the
DQI would remain a tool of limited empirical value because
in itself it can shed no light on two crucial issues:
first, whether high-quality deliberation has a greater
tendency to produce policy consensus; and, second, whether
the policies and decisions that emerge from deliberative
processes are in some sense superior to those arrived at
via more traditional methods.19 We will return to these
questions in a moment.
Finally in this section, it is worth mentioning the
entirely different means of identifying deliberation
proposed by Nanz and Steffek.20 They argue that in the
particular case of international governance institutions,
in which any kind of mass participation is impossible for 19 It should be emphasised that the authors of the DQI make no claim
that it can be used as anything other than a means to identify
authentically deliberative processes and institutions. They
themselves argue that the really interesting questions arise only
once we have a reliable means of identification in hand and that the
point of the DQI is to provide that and nothing more. See Bächtiger
et al., ‘The Deliberative Dimensions of Legislatures’, p.226.18
pragmatic reasons, we can nevertheless evaluate the
deliberative quality of decision-making processes by asking
if ‘there is a warranted presumption that public opinion is
formed on the basis of adequate information and relevant
reasons, and that those whose interests are involved have
an equal and effective opportunity to make their own
interests (and their reasons for them) known’.21 Nanz and
Steffek’s operationalisation of this question involves
assessing the degree to which civil society organisations
(CSOs) are able to access and influence the policy- and
decision-making procedures of international organisations.
They propose four criteria: the access granted to CSOs (no
access, observer status or the right to speak in meetings
and submit documentation); the degree of transparency and
access to information (no access to documentation, access
to background documentation or full access to background
and policy documents); the responsiveness of international
organisations to stakeholder concerns (CSO concerns and
positions not discussed at all, state actors justify
proposals with reference to concerns voiced by CSOs or CSO
concerns become part of the agenda); and finally the degree
19
to which policy processes are actively inclusive (whether
or not institutional arrangements are made to ensure the
inclusion of all relevant CSOs).
The feasibility-capacity stream
Serious doubts have been expressed about the capacity of
individuals to deliberate in a way that meets the criteria
established in normative theory. As Ryfe puts it, we need
to ask whether deliberative democracy can ‘work at the
fundamental level of human reasoning’.22 There are at least
three aspects to this question: psychological, sociological
and (for want of a better word) educational.
Rosenberg, drawing both on his own research and a review of
other relevant psychology literature, argues that ‘only a
20 Patrizia Nanz & Jens Steffek, ‘Assessing the Democratic Quality of
Deliberation in International Governance: Criteria and Research
Strategies’, Acta Politica, 40(3) (2005), 368-383.
21 Nanz & Steffek, ‘Assessing the Democratic Quality of Deliberation
in International Governance: Criteria and Research Strategies’, p.
370.
22 David M. Ryfe, ‘Does deliberative democracy work? A state of the
field’, Annual Review of Political Science, 8 (2005), 49-71, p.50.20
small minority of individuals demonstrate deliberative
rationality, that is the requisite capacity to reflect on
their preferences and organize them with regard to higher
order goals or overarching life-plans... Similarly only a
small minority of people demonstrates deliberative
reasonableness and thus takes the perspective of another
and makes arguments that are persuasive in his terms’.23 A
number of other authors argue that opinion or preference
formation is only infrequently a matter of systematic
reasoning, more often involving a rather less cognitively
demanding process of responding to ‘heuristic cues’24 or
assessing new information for its compatibility with
existing beliefs and values or conceptual ‘frames’.25
Individuals may also change or adopt new opinions and
preferences as a result of ‘groupthink’, a process in which
‘likeminded or cohesive individuals either mutually
23 Shawn Rosenberg, ‘The Empirical Study of Deliberative Democracy:
Setting a Research Agenda’, Acta Politica, 40(2) (2005), 212-224, p.
221.
24 Ryfe, ‘Does deliberative democracy work ? A state of the field’ ;
Hans-Peter Kriesi, ‘Argument-Based Strategies in Direct-Democratic
Votes: The Swiss Experience’ Acta Politica, 40(3) (2005), 299-316.21
reinforce their existing perspectives… or engage in strong
consensus seeking behaviour so that it overwhelms any
potentially dissenting issues’.26
Scepticism about deliberation has also arisen from work in
sociology. Three particular problems have been identified
that, a priori, seem likely to have an impact on the
possibility of deliberation. The first is that the capacity
to deliberate and the ability to have ideas and opinions
taken seriously is unevenly distributed among different
members of society. For example, Sanders argues that not
only do the education and cultural capital associated with
the ability to argue a case convincingly vary with income,
25 Shane J. Ralston, ‘Intelligently Designing Deliberative Health Care
Forums: Dewey's Metaphysics, Cognitive Science and a Brazilian
Example’, Review of Policy Research, 25(6) (2008), 619-630.
26 Simon Niemeyer & John Dryzek, ‘Intersubjective rationality: Using
interpersonal consistency as a measure of deliberative quality’,
paper presented at the European Consortium for Political Research
35th Joint Sessions of Workshops, Helsinki, 2007, p. 17. Available
at:
http://deliberativedemocracy.anu.edu.au/documents/ECPSRHelsinkiPaper
_NiemeyerandDryzek2007.pdf [Accessed May 12, 2009].22
but inequalities are so deeply ingrained in our societies
that the simple fact of being, say, a black woman means
that your opinion is likely to carry less weight in
discussion.27 Hence, the argument goes, rather than
permitting participation on an equal basis regardless of
sex, race, class and so forth, deliberation may actually
tend to reinforce existing structures of power and
inequality. A second problem, closely related to the first,
is that deliberation is itself a culturally loaded
procedure that privileges calm, dispassionate modes of
discourse over overtly emotive but (arguably) equally valid
forms of expression like storytelling or testimony.
Participants from social backgrounds or cultures
characterized by these less ‘rational’ modes of expression
are thus prevented from making their voices heard.
Finally, the deliberative emphasis on consensus and
compromise, the search for what unites rather than what
divides, puts at risk the expression of difference and
particularity. As Sanders puts it, ‘In settings where there
27 Lynn M. Sanders, ‘Against Deliberation’, Political Theory, 25(3) (1997),
347-376.23
are gross inequalities in power and status, calling for
compromise may be perilously close to suppressing the
challenging perspectives of marginalised groups’.28
If these psychological and sociological critiques are well-
founded – and the question is clearly an empirical rather
than a theoretical one – then the basic conditions for
deliberation cannot be met. Even where an apparently
consensual decision emerges from discussion, it cannot be
assumed to have the privileged epistemic and normative
status that in principle accrues to the outcomes of
deliberative processes. The principal difference between
the psychologically- and sociologically-grounded critiques,
other than the fact that there is rather more concrete
evidence for the former, is that the psychologists are more
optimistic about the possibility of correcting the problem.
Although Neblo has suggested that there is ‘under-
appreciated room for mutual accommodation among
deliberative theorists on questions of emotion, rhetorica
and alternate communication forms’,29 most of the
sociological critics of deliberation seem unwilling to 28 Sanders, ‘Against Deliberation’, p. 362.
24
accept that any conceivable intervention would correct the
difficulties they identify.30 On the other hand, Delli
Carpini and his colleagues, argue that although the
research evidence shows that successful deliberation is
highly context-dependent and ‘rife with opportunities for
going awry’,31 it also provides a great deal of indirect
support for the democratic potential of deliberation.
Rosenberg takes the research evidence as a challenge to be
met rather than an immovable obstacle:
if deliberations were designed with the limitations of
the participants in mind, two goals might be
accomplished. First, more reasoned and just processes
and outcomes might be obtained. Second, the
deliberation might provide a context for the further
development of the participants’ existing deliberative
capacities... In either case, the consideration of the
nature and design of deliberative institutions
necessarily shifts away from the more typical focus on
establishing conditions and processes that free
individuals to do what they already can. Instead
25
attention turns to establishing conditions and
processes that guide individuals so that they can
achieve a potential that they possess but have not
realized.32
Work carried out by researchers at the Australian National
University’s deliberative democracy research group provides
further important evidence both that deliberation is a
distinctive socio-cognitive process and that it has the
29 Michael A. Neblo, ‘Family Disputes: Diversity in Defining and
Measuring Deliberation’, Swiss Political Science Review, 13, 527-557 (2007),
p. 535.
30 See for example Chantal Mouffe, ‘Deliberative Democracy or
Agonistic Pluralism?’, Social Research, 66(3), 745-758; Sanders,
‘Against Deliberation’; Elaine Stratford, Denbeigh Armstrong &
Martina Jaskolski, ‘Relational Spaces and the Geopolitics of
Community Participation in Two Tasmanian Local Governments: A Case
for Agonistic Pluralism?’, Transactions of the Institute of British Geographers,
28(4) (2003), 461-472.
31 Michael X. Delli Carpini, Fay L. Cook & Lawrence R. Jacobs, ‘Public
deliberation, discursive participation and citizen engagement: A
Review of the Empirical Literature’, Annual Review of Political Science, 7(1)
(2004), 315-344, p. 328.26
effects predicted by normative theory for the reasons that
these effects are expected.33 As we saw above, one of the
two most basic conceptual criteria for deliberation is
reason-giving. Both the epistemic value and normative
legitimacy of decisions reached via deliberative methods
arise from consensus on the underlying reasons for taking
some collective action. On this basis, the ANU researchers
argue that an increase in what they call ‘intersubjective
consistency’ can be used as a measure of the effectiveness
of deliberation. Intersubjective consistency occurs when a
group of individuals not only have common preferences, but
also share the beliefs and values (subjectivity) that give
rise to those preferences. While a convergence of
preferences alone may be the result of non-deliberative
social processes, where the subjective reasons underlying
preference formation also converge as a result of
discussion and debate we have strong evidence that
deliberation has occurred. The ANU researchers have been
able to show that intersubjective consistency does indeed
32 Rosenberg, ‘The Empirical Study of Deliberative Democracy: Setting
a Research Agenda’, p. 222.27
increase as a result of deliberation.
The third aspect of the feasibility-capacity research
stream is concerned not with the inherent intellectual or
psychological capacities of participants in deliberation,
but with their level of education and their knowledge of
the issues under discussion. A potential problem that is
frequently raised is that the technical complexity of
certain policy issues is such that most participants will
be unable to grasp them sufficiently well for meaningful
deliberation to be possible. For example, in a discussion
of the ‘transition’ economies of central and eastern
Europe, Przeworski argues that market economic reforms ‘are
based on a model of economic efficiency that is highly
technical. They involve choices that are not easy to
explain to the general public and that do not always make
sense to popular opinion’.34 However, the evidence that non-
expert participants in deliberation are capable of making
sense of complex technical issues is very strong.35
Esterling and his colleagues also found that participants
in deliberation use the process precisely to become
28
informed about policy issues,36 while Ryfe suggests that
participants who are knowledgeable about a particular issue
may actually be less able to deliberate effectively on that
issue than those who are not. Ryfe argues that expert
participants tend to be ‘more rigid in their thinking, less
tolerant of others’ views and, perhaps most importantly,
more adept at rationalizing their own’.37
The institutional-environmental stream
The impact of four main types of contingent factor external
to deliberative processes has been considered in this
stream in the literature: institutional characteristics
(whether the institutions in question are specifically
designed to promote deliberation or not); the features of
the social and political environment; the initial political
and value commitments of participants; and the nature of
the issues under discussion.
There is little consensus on what institutional features 33 Niemeyer et al. ‘Achieving Success in Large Scale Deliberation: An
Analysis of the Fremantle Bridge Community Engagement Process’;
Niemeyer & Dryzek, ‘Intersubjective rationality: Using interpersonal
consistency as a measure of deliberative quality.’29
make for an effective deliberative forum, the range of
possible designs being extremely wide. As Fung puts it,
‘The menu of institutional alternatives is far richer than
the dichotomy between representative and participatory
democracy supposes, and most of the items on that menu
remain empirically and normatively unexplored’.38 A detailed
assessment of the pros and cons of different types of forum
is beyond the scope of this review, but we can at least
indicate the principal dimensions of the problem. In the
literature, discussion focuses on the aim or purpose of the
forum, the integration of the forum into the larger
political process, the selection of participants (including
professional advocates or other stakeholders), the
organisation of participants within the deliberative
process and the role played by experts.
An issue raised by several authors is the ultimate aim or
purpose of the deliberative forum. For example, Setälä et
al distinguish between processes that require participants
to come up with some kind of collective conclusion and
those that allow participants to express an individual
30
opinion at the end of the process by voting or responding
to an attitude survey or poll.39 This is related to the
issue of what Grogan and Gusmano call the ‘purpose and
style’ of deliberation.40 Citing the work of Button &
Mattson, they outline four possible models: educative,
intended to provide participants with information and
knowledge; consensual, aimed at finding agreement on an
issue, value or plan for the future; activist/instrumental,
leading to political or legislative action; and conflictual,
emphasising the provision of the widest possible space for
the expression of different points of view.41
Related to the question of what deliberative forums are for
is the issue of the connection between the decision or
policy of a forum and the larger political process. Goodin
and Dryzek list eight different ways in which deliberative
forums can be integrated into politics, from actually
making policy to simply informing public debates.42 Dryzek
and Tucker suggest that the output of deliberative forums
can be deployed in integrative or managerial ways or as
advocacy.43 In the first case, the output of deliberation ‘is
31
integrated into the established policy making structure,
and is designed to help integrate informed public opinion
and key actors into that structure’.44 In the second case,
the deliberative forum is ‘used as a tool through which key
policy makers … produce an alternative representation of
34 Cited in Dryzek, ‘Democratization as deliberative capacity building’,
p.13.
35 See for example John Dryzek & Aviezer Tucker, ‘Deliberative
innovation to different effect: Consensus conferences in Denmark,
France and the United States’, Australian National University, 2005.
Available at:
http://deliberativedemocracy.anu.edu.au/documents/DryzekandTucker200
5.pdf [Accessed May 10, 2009]; Kaspar M. Hansen, Vibeke N. Andersen,
‘Deliberative democracy and the deliberative poll on the euro’,
Scandinavian Political Studies, 27(3) (2004); Giorgos Kallis, Dionyssia
Hatzilacou, Alexandra Mexa, Harry Coccossis & Eleni Svoronou,
‘Beyond the manual: Practicing deliberative visioning in a Greek
island’, Ecological Economics, 68 (2009), 979-989; Carolyn M. Hendriks,
John Dryzek & Christian Hunold, ‘Turning up the heat: partisanship
in deliberative innovation, Political Studies, 55 (2007), 362-383.
36 Kevin M. Esterling, Michael A. Neblo, David M. Lazer, ‘Means,
Motive, & Opportunity in Becoming Informed About Politics: A
Deliberative Field Experiment’, paper presented at the annual 32
informed public opinion to contrast with the uninformed
skepticism of the mass public’ on the issue in question.45
In the third case, the results of deliberation are
mobilized by civil society actors in lobbying and advocacy
work.
While participation in ‘town meeting’ style deliberative
forums46 tends to be entirely open or self-selecting, most
other types of forum adopt a representativity rule for the
selection of participants. Deliberative polls, for example,
aim to attract a set of participants that, while otherwise
randomly selected, is as statistically representative of
the relevant population as possible.47 Other types of forum
operate on the basis of descriptive representation or
‘social mirroring’ as Hendriks calls it.48 As Goodin and
Dryzek put it, what this means is that ‘the diversity of
meeting of the Americal Political Science Association, Chicago,
2007. Available at:
http://www.hks.harvard.edu/netgov/files/png_workingpaper_series/PNG0
7-006.pdf [Accessed June 8, 2009].
37 Ryfe, ‘Does deliberative democracy work ? A state of the field’, p.
5.33
social characteristics and plurality of initial points of
view in the larger society are substantially present in the
deliberating mini-public. Social characteristics and
viewpoints need not be present in the same proportions as
in the larger population’.49 Descriptive representativity
arguably includes those situations in which certain social
groups are represented by professional advocates who, while
not elected, have some warranted claim to be able to speak
on behalf of those groups. Finally, there is electoral
representativity which, obviously enough, involves
participants who are elected by and are accountable to
certain sections of society. Although no conclusions seem
to have been drawn about whether statistical or descriptive
representativity makes for better deliberation, there is
some agreement that whether or not participants are
‘partisan’ is an important variable. The distinction
between partisan and non-partisan participants turns
essentially on whether or not they have relatively open
38 Archon Fung, ‘Democratic Theory and Political Science: A Pragmatic
Method of Constructive Engagement’, American Political Science Review, 101(3)
(2007), p. 445.34
preferences on the issue under discussion. As to the
question of whether partisanship is a good or a bad thing,
Hendriks et al found that ‘[w]hile the partisan
participants [in their study] accepted that deliberation
requires willingness to adjust preferences, they failed to
translate this into action and held on tightly to their
positions. Our case suggests that partisans might be
morally committed to the idea of deliberation, but they
struggle to put it into practice’.50 On the other hand,
Grogan and Gusmano argue that while it is crucial to
involve an appropriate range of descriptively 39 Maija Setälä, Kimmo Grönlund & Kaisa Herne Citizen deliberation on
nuclear power: A Comparison of two decision-making methods.
University of Berne Center for Interdisciplinary Deliberation
Studies, 2004. Available at:
http://www.bids.unibe.ch/unibe/rechtswissenschaft/oefre/bids/content
/e3409/e3822/e3824/linkliste3828/Setl:Grnlund:Herne.pdf [Accessed
June 7, 2009].
40 Colleen M. Grogan & Michael K. Gusmano, ‘Deliberative democracy in
theory and practice: Connecticut's medicaid managed care council’
State Politics and Policy Quarterly, 5(2) (2005), 126-146.
41 Grogan & Gusmano, ‘Deliberative democracy in theory and practice:
Connecticut's medicaid managed care council’, p. 132.35
representative but otherwise non-partisan participants,
this should not be seen as an alternative to the presence
of professional advocates for the interested social groups
because these professional partisans are more inclined to
challenge the views of experts.51
The organisation of participants within the deliberative
process – their allocation to different working groups, the
relationship between these groups and any plenary
discussion sessions and so forth – seems in most cases to
be of relatively little interest. However, two cases are
worth mentioning. Davies and Burgess report on a
deliberative forum dealing with the allocation of organs
for transplantation in which participants were segregated
by both sex and socio-economic status.52 The authors found
significant differences in the discursive practices of the
42 Robert E. Goodin & John S. Dryzek, ‘Deliberative Impacts: The
Macro-Political Uptake of Mini-Publics’, Politics & Society, 34(2) (2006),
219-244.
43 Dryzek & Tucker, Deliberative innovation to different effect:
Consensus conferences in Denmark, France and the United States.
45 Dryzek & Tucker, op. cit., p. 22.36
male and female groups, in particular with respect to their
interaction with the healthcare professionals advising the
forum. In their study of educational segregation,
Mendelberg and Oleske report on a series of open-
participation town meetings which were spontaneously
segregated by race.53 This segregation proved to be a
serious obstacle to deliberative interaction as the
arguments put forward by participants remained within the
bounds of types of discourse which, while largely held to
be valid within each racial group, were not so recognized
outside them or, worse, were interpreted as coded forms of
racism.
There is rather less research on the institutional and
procedural characteristics that are associated with
deliberative forms of interaction in ‘normal’ democratic
political contexts – as opposed to forums specifically
designed to permit deliberation. Nevertheless, the DQI and
similar approaches have been able to shed some light on the
relationship between the nature of discussion and debate
that takes place within legislative bodies, different types
37
of political institutions and different aspects of the
political environment. Bächtiger and his colleagues found
that five factors were of relevance.54 Presidential systems
of government, in which the executive and legislative
functions are strictly separated, a political culture that
emphasizes consensus over competition, debate in second
chambers, non-public debate and low issue polarisation were
all found be associated with higher quality discourse.
Falling somewhere between the DQI and the Australian
National University’s intersubjective consistency approach
we find the work of Karlsson on the European Convention,
the representative group that prepared the text of the ill-
fated European constitution.55 On the basis of interviews
with participants in the Convention, Karlsson attempts to
determine to what extent participants went into the process
with open minds about what the outcomes might be; the
degree to which they actually did change their views in the
course of the process; and the degree to which this change
was due to a rational engagement with arguments made by
other participants rather than a consequence of, for
38
example, threats or political expediency. He found that the
participants’ widely-shared view that the process was more
deliberative than negotiated was consistent with their
accounts of the development of their opinions and their
narratives of how the Convention had unfolded.
44 Dryzek & Tucker, op. cit., p. 15.
46 Mansbridge, Beyond Adversary Democracy; Mandelberg & Oleske, ‘Race &
Public Deliberation’.
47 James S. Fishkin & Robert C. Luskin, ‘Experimenting with a
Democratic Ideal: Deliberative Polling and Public Opinion’, Acta
Politica, 40(3) (2005), 284-298; Robert C. Luskin, James S. Fishkin &
Roger Jowell, ‘Considered opinions: Deliberative polling in
Britain’, British Journal of Political Science, 32 (2002), 455.
48 Carolyn M. Hendriks, ‘On inclusion and network governance: the
democratic disconnect of Dutch energy transitions’, Public Administration,
86(4) (2008), 1009-1031.
49 Goodin & Dryzek, ‘Deliberative Impacts : The Macro-Political Uptake
of Mini-Publics’, p. 221.
50 Hendriks et al, ‘Turning up the heat: partisanship in deliberative
innovation’, p. 370.
51 Grogan & Gusmano, ‘Deliberative democracy in theory and practice:
Connecticut's medicaid managed care council’.39
The issue of open-mindedness and the possibility of
preference change has also been examined by Hendriks and
her colleagues in their work on the effect of partisanship
on deliberation.56 ‘Partisans’ are those who possess firm
and established opinions on an issue, or who are active
advocates for a particular group or point of view. The
researchers studied existing forums for discussion, debate
and decision-making, dividing them into those in which
partisan participants were directly involved in
deliberation and those in which they were merely invited to
present and defend their position before panels of ‘lay’
citizens. The different types of forum were compared on the
basis of their deliberative capacity, the legitimacy of
their procedures and decisions in the view of key
stakeholders and the media, and the political impact of
their findings. Arguably the most important finding of the
52 Gail Davies & Jacquelin Burgess, ‘Challenging the ‘view from
nowhere’: citizen reflections on specialist expertise in a
deliberative process’, Health & Place, 10 (2004), 349-361.
53 Mendelberg & Oleske, ‘Race & Public Deliberation’.
54 Bächtiger et al, ‘The deliberative dimensions of legislatures’.40
research was that the partisan forums had lower
deliberative capacity than the non-partisan forums: ‘While
the partisan participants accepted that deliberation
requires willingness to adjust preferences, they failed to
translate this into action and held on tightly to their
positions. Our case suggests that partisans might be
morally committed to the idea of deliberation, but they
struggle to put it into practice’.57 The partisan forums
were also perceived as having lower legitimacy, although
this may have been due to the means of measuring legitimacy
used in the research. As the authors put it, ‘Legitimacy in
the partisan case turned largely on whether the
deliberations produced the preferred outcome for the
organizations with an interest in the case’.58 On the other
hand, the authors report that these same organisations
criticized the non-partisan forums on the grounds that the
participants did not properly appreciate the ‘science’ of
the issues, nor what it was like to be ‘affected’.
Finally in this section we should mention the issue
dimension, which is to say the type of policy issue or 55 Karlsson, ‘Deliberation at the European Convention’.
41
decision under discussion. We have already considered the
question of the scientific or technical complexity of the
subject matter of deliberation. Beyond this, the literature
on the topic is not extensive. Few if any researchers have
been able to go beyond suggesting that non-contentious
issues lend themselves rather more easily to high quality
deliberation than, say, stem cell research, and that the
more concrete the issue – the closer the deliberative forum
is to having to make an actual final decision – the more
likely it is that participants will adopt rather entrenched
and inflexible positions. For example, Papdopoulos, citing
work carried out by Klaus Eder, reports that ‘Research on
deliberative institutions of environmental policy in
Germany ... shows that their contribution to consensus-
building is higher when concrete decisions are not at
stake, and confirms that the initial level of polarisation
should not be too high’.59
56 Hendriks et al, ‘Turning up the heat: partisanship in deliberative
innovation’.
57 Hendriks et al, op.cit., p. 370.
58 Hendriks et al, op. cit., p374.42
Methodological approaches
In this final section of our review we will give a brief
overview of the methodological approaches adopted by
empirical deliberation researchers. Once again we can
identify a number of different – but non-exclusive –
streams in the literature. First of all there are those
approaches based on the direct analysis of discursive
practice ‘as it happened’. Steiner et al’s DQI is perhaps
the best example in this category, although we have also
mentioned the work of Holzinger and Conover and Searing.60
In all three of these cases, the researchers apply formal
content analysis to the transcripts of discussions, coding
different types of speech or the attitudes manifested by
participants in order to carry out some kind of statistical
analysis. Other researchers also make use of transcripts of
discussions but without using formal analytic techniques.61
A second stream in the literature involves the discussion
59 Yannis Papadopoulos, ‘Towards some research questions on the merits
and limits of deliberative policy-making’, paper presented at the
conference Empirical Approaches to Deliberative Politics, European University
Institute, Florence, 2004, p. 9.43
and analysis of participants’ post-deliberation reflections
on the process and outcomes of deliberation. This approach,
generally involving semi-structured interviews, is clearly
the most widely used, whether as a principal research
method or in support of other more formal techniques.
Karlsson’s work on the European Convention, for example, is
based exclusively on interviews, while Niemeyer and his
colleagues combine post-discussion interviews with
participants with both informal content analysis of
discussion transcripts and survey techniques.62
Work involving survey methods represents a third
substantial stream in the literature. Deliberative polling
is the best known and best established of the survey-based
empirical approaches, functioning essentially via a
60 Steiner et al, Deliberative Politics in Action ; Holzinger, ‘Context or
conflict types’ ; Conover & Searing, ‘Studying ‘Everyday Political
Talk’ in the Deliberative System’.
61 See for example Davies & Burgess, ‘Challenging the ‘view from
nowhere’’; Kallis et al, ‘Beyond the manual’.
62 Karlsson, ‘Deliberation at the European Convention’; Niemeyer et al,
‘Achieving success in large-scale deliberation’.44
relatively straightforward comparison of participants’
expressed preferences before and after deliberation. More
recently, the Australian National University research group
has made a good case for the use of Q-methodology.63 This is
a type of factor analysis performed on subjects’ rankings
of their degree of agreement or disagreement with 30 to 60
statements about an issue. The Q factor analysis provides a
picture of each subject’s subjective beliefs at a
particular point in time and so permits a comparison not
just of preferences before and after deliberation, but the
reasons for which those preferences are held.
Finally, we have those methodological approaches that focus
on the analysis of institutional characteristics or the
structure of deliberative processes and the effect of
contingent environmental factors. This type of approach is
most frequently found in conjunction with other techniques
aiming to evaluate the quality of discursive practices or
the outcomes of deliberation, although the work of Nanz and
Steffek is an unusual example of an attempt to measure the
63 Niemeyer & Dryzek, ‘Intersubjective Rationality’; Niemeyer et al, op.
cit.45
democratic quality of institutions as an end in itself.64 In
most of these cases, the analysis involves the
categorisation of institutional or environmental factors
into two or more types that might be expected to have a
different effect on discursive practice. Hence we have, for
example, Dryzek’s categorisation of political systems
according to their openness to influence from organized
activist groups or Hendriks’s division of deliberative
forums into partisan and non-partisan.65
Conclusions
In this review we have described the principal theoretical
and technical issues involved in empirical research on
deliberative democracy and decision-making. We have seen
that, insofar as it addresses the empirical conditions for
deliberation, existing research can be divided into three
streams which we have called the behavioural-procedural,
feasibility-capacity and institutional-environmental 64 Nanz & Steffek, ‘Assessing the Democratic Quality of Deliberation in
International Governance’.
65 Dryzek & Tucker, ‘Deliberative Innovation to Different Effect’;
Hendriks et al., ‘Turning up the Heat’.46
streams. Notable by its almost complete absence, however,
is any work that aims to evaluate the technical
effectiveness of deliberative policy- and decision-making
relative to other types of process, democratic or
otherwise. Although a number of authors pose the question,66
it is only to note that it has not (yet) been answered.
This is all the more surprising bearing in mind that one of
the most frequent claims in the normative literature is
that deliberation has a strong epistemic or truth-tracking
potential. In principle, then, it ought to lead to the best
possible policy outcomes, although as Habermas puts it,
‘Whether deliberation does indeed introduce an epistemic
dimension into political will-formation and decision-making
is, of course, an empirical question”.67
47
66 See, for example, Patricia Fitzpatrick, A. John Sinclair & Bruce
Mitchell, ‘Environmental impact assessment under the Mackenzie
Valley Resource Management Act: Deliberative democracy in Canada's
North?’ Environmental Management, 42(1) (2008), 1-18, p.16; David M.
Ryfe, ‘The practice of deliberative democracy: a study of 16
deliberative organizations’, Political Communication, 19 (2002), 359-377,
p.370.
67 Jürgen Habermas, ‘Political Communication in Media Society: Does
Democracy Still Enjoy an Epistemic Dimension? The Impact of
Normative Theory on Empirical Research’ Communication Theory, 16 (206),
411-426, p.413.48