Online Forums and Deliberative Democracy

Davy Janssena and Raphael KiesbaDept. Politieke Wetenschappen, Universiteit van Antwerpen, Prinsstraat 13, 2000 Antwerpen,

Belgie.

E-mail: davy.janssen@ua.ac.bebEuropean University Institute, Florence, Italy.

E-mail: kies@ieu.it

The Internet is rapidly becoming a part of the everyday lives of a majority of peoplein the Western world. People perform various activities on the Internet and one ofthem is discussing politics and society in so-called online forums. In this article, wepresent an overview of some of the empirical research that evaluates the quality ofpolitical conversations in online forums. In the first section, we distinguish researchon Usenet groups, web-based political forums and e-consultation forums, and discusssome its findings. In the second section, we elaborate three original categories ofvariables that attempt to explain differences observed in the quality of deliberation.The third, more extensive section deals with methodological issues. It discusses theoperationalisation of deliberative quality and the application of a set of criteria forthe idealized public sphere to online conversations. In the conclusion, we presentsome objections to the previous research and offer some ideas for a morecomprehensive approach to online forum analysis.Acta Politica (2005) 40, 317–335. doi:10.1057/palgrave.ap.5500115

Keywords: internet; online forums; measuring deliberation

Research on Online Political Forums

In this section we present some of the literature on online forum analysis byusing the following classification: research on Usenet groups; research onpolitical forums; research on e-consultation forums; research on particularonline deliberative experiments. Because of lack of space we do not discussstudies in the fourth category here: they are mostly case study reports onspecific experiments.1

Usenet groups

As Davis (1999) defines it, ‘Usenet is a computer conferencing networkallowing any user to read and even post messages on an electronic bulletinboard’. These systems of interaction were implemented in the 70’s, but theiruptake surged during the 1990s: ‘by 1995, an estimated 180,000 users were

Acta Politica, 2005, 40, (317–335)r 2005 Palgrave Macmillan Ltd 0001-6810/05 $30.00

www.palgrave-journals.com/ap

posting nearly one million messages daily’ (Davis, 1999, 150). The existingresearch on political newsgroups can be divided into two broad categories. Thefirst contains studies of Usenet groups that are affiliated with political parties,coalitions of parties and/or candidates. In this context Bentivegna (1998) hasanalysed four newsgroups: two were affiliated with political parties and twowith a coalition of parties. The second category of studies deals with more issuerelated newsgroups. Davis (1999) analysed three Usenet groups that concerneda variety of topics: the politics of Clinton, the US Constitution, and radical-leftideology. Schneider (1997) carried out an extensive empirical investigation ofa Usenet newsgroup on abortion. More recently, Dumoulin (2003) analysedthree online newsgroups in Canada. Two were based on topics of politicalinterest (Western thoughts of separation and Avant-Garde Quebec) and one ona topic of social interest (gay rights). Finally, Wilhelm (1999) analysed 10political newsgroups chosen at random, six from Usenet newsgroups and fourfrom the commercial provider AOL.

Concerning the findings, the research suggests that Usenet groups arediscursive contexts that do not promote qualitative forms of deliberation. Theyare spaces that do not encourage reciprocal exchanges between citizens.Wilhelm observed that the discursive dynamic was characterised by ‘the self-expression of monologue, without in large measure the listening, responsive-ness, and dialogue that would promote communicative actions’ (Wilhelm,1999, 98). Dumoulin defines this phenomenon as ‘interactive monologues’.Additionally, several authors (Davis, 1999; Wilhelm, 1999; Sunstein, 2001)observed that Usenet groups favour the encounter of similar opinions and, as aconsequence, lead to polarisation of opinions. Additionally, Usenet conversa-tions are generally not considered to be deliberative, also since crossposting(i.e. the sending of a message to different newsgroups) (Bentivegna, 1998) and‘flaming’ (i.e. the use of abusive language) tend to be strongly present (Dutton,1996; Davis, 1999), in particular for topics that are related to personal issues(Dumoulin, 2003).

Political forums

Political forums are spaces of discussion that are hosted on the World WideWeb and that use a web-based visualization. We distinguish three contexts inwhich political forums have been analysed. The first is the forum contained bya local website, whether public, such as the city of Santa Monica (Dutton,1996; Docter and Dutton, 1998) or the city of Hoogeveen (Jankowski and vanOs, 2002), or private, such as the Minnesota e-democracy experience (Aikens,1997; Jensen, 2003b). The second are the forums hosted at the national level bygovernment institutions, such as the Citizenspace forum in the UK (Colemanet al., 2002) or Nordpol in Denmark (Jensen, 2003a). The third are the forums

Davy Janssen and Raphael KiesOnline Forums

318

Acta Politica 2005 40

hosted by traditional media organizations such as newspapers (Schultz, 2000;Tanner, 2001). We will limit our analysis here to national institutional forumsand media organizations’ forums.

Citizenspace is a good example of a national institutional forum. Its purposeis to enable citizens to enter into an interactive relationship with thegovernment. The Hansard Society, however, suggests that the debates werenot as constructive as expected and that the government representatives did notparticipate (Coleman et al., 2002). In fact, the forum ‘lacked a clear purpose orconnection to Government policy making’, and worse, ‘it proved an outlet forill-informed opinion, prejudice or abuse’ (Coleman et al., 2002, 12).

Regarding the forums hosted by media organizations, Schultz (2000)observed a forum hosted by the New York Times, while Tanner (2001)analysed one hosted by a Chilean newspaper. Schultz based his analysis ona survey of the journalists and users of the forum. He found that journalistsnormally do not participate since they lack time. However, they do sometimesread the forums they consider constructive. As to the users, the survey indicatesthat they tended to be highly educated and male. Tanner focused on differentaspects: the quality of interactions and the social role function. To this end shefocused on a specific forum where Chileans from around the world discussedthe October 1998 arrest of ex-dictator Augusto Pinochet and the legacy of themilitary government. Again, participants tended to be highly educated andmale. However, even if the forum did not match the ideal level of deliberation— she underlined problems of crossposting, and of total absence of agreement— Tanner considered that such a forum contributed to the reconciliationprocess in Chile through the development of public opinion and the creationof a collective memory about the past event. The fact that an importantproportion of messages corresponded to ‘testimonial messages’ of theexperiences participants had with the dictatorship is, according to the author,indicative of this process.

E-consultation forum

E-consultation forums are some of the most institutionalized procedures toallow online participation of citizens in the political process. They refer tothe use of the internet to disseminate to the wider public, experts and interestgroups, developments in a policy field and invite them to respond. Therationale behind the promotion of e-consultation techniques is to encouragethe general public, interest groups and experts to participate in the decision-making process (Trechsel et al., 2004). E-consultation can take a variety oftechnological formats and can focus on very different topics: from questionsrelated to urban planning (Monnoyer-Smith, 2004) to e-consultation on

Davy Janssen and Raphael KiesOnline Forums

319

Acta Politica 2005 40

prospective bills (Beierle, 2002; Coleman et al., 2002) or on ongoing debatesduring the municipal council meetings.

The Hansard society (Coleman et al., 2002) provides a comparative study often e-consultation forums in the UK,which offers several interesting findings.It reveals that the rate of participation in these consultative forums varied a lot.Regarding the profile of participants, and in particular the gender balance,the study found that ‘gender balance is very much topic-related and reflectsthe more traditional divisions of interest between men and women’ (Colemanet al., 2002, 7). As to the quality of the forum, findings were mixed. Thepositive aspects are that most messages were relevant in the limited sense thatthey were not off-topic. Less positive impressions, however, concerned thequality of the debates, since ‘most participants in forums posted messageswhich simply stated their opinions’ and ‘very few participants in the forumseemed to use facts in their deliberation’ (Coleman et al., 2002, 8). Further, thestudy reveals that few government officials or politicians participated in theforums eventhough most messages were directed to them. Concerning thepolitical impact of the forum, the promoters confirmed that it was nonexistentor limited. The analysis concluded: ‘what is the value of inviting citizens todiscuss a policy or Bill if nobody at the other end is going to take any notice ofthem?’ (Coleman et al., 2002, 10). It would, however, be a mistake to considerthat all e-consultation forums are a failure from the point of view of theirquality and impact. The case of Commbill.net, the first UK e-consultationforum on a draft Bill, is indicative of the fact that if an e-consultation forum iswell organised, by offering, for instance, relevant information and updatedsummaries, it can be successful in terms of its impact (Coleman et al., 2002). Infact, the Committee that was established by both Houses of Parliamentreferred to the e-consultation forums several times and two of its key policyrecommendations came directly from evidences presented via the forum.

This initial analysis of the literature suggests that the basic distinction weproposed (Usenet, political forum and e-consulation forum) is insufficient forfully explaining differences in online deliberation. There are, for instance, toomany exceptions and contradictions to simply consider e-consultation contextsas ‘good’ for deliberation and newsgroup contexts as ‘bad’. Our intention inthe next section is to complement these preliminary findings by focusing onmore fine-tuned variables for explaining deliberative variations.

Variables Explaining Differences in Online Deliberation

Our typology is based on three general categories that correspond to threeavenues in which the research could advance in order to get a betterunderstanding of the online deliberation: (i) the communicative structure of the

Davy Janssen and Raphael KiesOnline Forums

320

Acta Politica 2005 40

discussion space, (ii) the political culture and ideology and (iii) the ‘weak’ and‘strong’ distinction. It is important to stress that these variables will interact,which means that the established level of deliberation will necessarily resultfrom a combination of them.

Communicative structure of discussion space

The communicative structure, as we define it, corresponds to the technical andorganizational architecture of the discussion space. Regarding the technicalarchitecture, it is fundamental to distinguish the real-time discussion spaces(chat rooms) from the asynchronous online discussion spaces that do not havetime constraints (email list; newsgroups; Bulletin boards; forums). It isgenerally recognized that the former are spaces of encounter that attract ‘smalltalk’ and jokes, while the latter constitutes a more favourable place for theappearance of some form of rational–critical form of debate since it allowsparticipants to spend more time to think and justify their interventions.

The second constitutive element of the communicative structure is the waythe online discussion spaces are organized and ruled. According to ourobservations, two organizational issues are particularly relevant as far asdeliberation is concerned: the request of identification and the presence of amoderator.

The issue of identification is a controversial one: on the one side it isassumed that the possibility to hide the real identity, which is the case for mostof the online discussion spaces, can have a positive impact on deliberationbecause participants will feel more free to express their real opinions. In otherwords, online discussion spaces where no identification is required could havea ‘disinhibiting effect’ (Dutton, 1996). The other side, however, argues thatparticipants should be identified in order to act as ‘responsible’ actors in thedebate: politics would be too serious to be discussed by ‘unidentified persons’(Maldonado, 1997). A research that we are conducting on the forum of websiteof the ‘Italian Radicals’, one of the most frequented and interesting politicalforum in Italy,2 suggests that indeed the identification of the participants is afundamental element for explaining the quality and the persistency of apolitical debate.3

The question of moderation tends to be controversial as well. Some arguethat on the Internet no moderation should be allowed because the ‘cyberspaceis a space of freedom’ while others consider that the moderation should beallowed in order to guarantee a certain structure and continuity in the debate.The impact of the moderation on deliberation will obviously depend on thetype of moderation implemented. The moderator can be a ‘censor’ — forexample by removing opinions that are at odds with the main ideology of thediscussion space — or he can be ‘promoter of deliberation’ by, for example,

Davy Janssen and Raphael KiesOnline Forums

321

Acta Politica 2005 40

implementing a system of synthesis of debate, by giving more visibility tominority opinions, by offering background information related to the topicsetc. Our research on the online community of the ‘Italian Radicals’ suggeststhat moderation based on censure can be detrimental for maintaining theliveliness and the quality of an online political debate since it often leads topolemical debates and discourages users from participating.4

The issues of identification and moderation should not hide otherorganizational issues such as the ownership of agenda setting or the accessibilityof the forum. The first indicates whether the possibility for initiating anddefining the topics of the debate is decentralized or not (can forum participantsinitiate their own topics or not) and the latter indicates who can participate inthe forum (total openness or not). Even if no specific research is conducted forevaluating their specific impact on deliberation, it is essential to keep thempresent as possible ‘intervening variables’.

Political culture and ideology

By referring to the criteria ‘political culture and ideology’ we imply that thesocio-political contexts in which the online discussion space is introduced isan important factor for explaining divergences observed in the deliberationquality.

At the broad level, we suggest that the quality of deliberation could beaffected by the geographical and cultural zone in which the online debate takesplace. Deliberation in online discussion spaces is unlikely to be the same instrongly divergent cultural zones such as the Asiatic, European or Arabicworld. It is also likely that differences will be observed among the countries ofthe same cultural region and within those countries. Combining inter-culturalstudies with their impact on online deliberation would constitute a fascinatingtopic for further research. We also suggest that the quality of deliberationcould be affected by the category of political actors hosting the debates(Institutional sites, NGOs sites, News sites, political parties sites, trade unionsites). The fact that each type of political actor hosts different kinds ofparticipants and has different interests and aims suggests that the debates intheir online public space are dissimilar. The question we raise is whether somesocial actors are more likely than others to host debates that tend to bedeliberative.

At a more narrow level we imply that the political beliefs and ideology (right,left, ecological, neutral, etc.) of the actors hosting the debates could alsoinfluence the level of deliberation of the debates. In this sense, Davis observedthat there were more personal attacks and flames in political discussion spaceswith extreme ideologies (such as the radical left Usenet group) than in forumsthat had a more balanced ideology (such as Clinton Usenet group) (Davis,

Davy Janssen and Raphael KiesOnline Forums

322

Acta Politica 2005 40

1999). Similarly, in a research we conducted last year we argued that the factthat a minor political party (Italian radicals) had a much greater and moredeliberative participation than the two major Italian political parties(Democratici di Sinistra and Forza Italia) was to a large extent, due to adifference of political culture: the ‘Italian Radicals’ have a much strongerparticipative culture than the other two political parties (Kies, 2003).

Finally, at a basic level, the literature suggests that the topic of the debate isa relevant variable for explaining differences observed in deliberation. It isa relevant variable for explaining variation in participation and in genderdistribution. Jensen (2003a) found that in an institutional forum citizens(nordpol.dk) are eager to engage in discussion when topics are related to theirdaily lives. Coleman (Coleman et al., 2002), based on a comparison of teninstitutional forums (in Scotland, Wales and UK), observed that the more thediscussion topic was related to problems of everyday life and personalexperience, the more women participated in the debates. This would reflectaccording to the authors: ‘the more traditional divisions of interest betweenwomen and men’ (Coleman et al., 2002). The topic discussed may alsoinfluence the content and the quality of the debates. In this context the analysisof Dumoulin (2003) is interesting. He compared three Canadian forums thatdiscussed different subjects: two dealing with topics of political interest(Western thoughts on separation and avant-garde Quebec) and one focused ona problem of social interest (gay rights). The author found, for instance, thatflames and moralizing discourses were particularly present in the gay rightsforum, which, according to him, was due to the fact it was a topic dealing with‘personal values’. He also noticed that the forums tended to be different fromthe point of view of their homogeneity (measure of plurality of opinions): ‘gayrights’ and ‘Western thoughts on separation’ had a debate that was pluralisticwhile, on the contrary, the debates within ‘Avant-Garde Quebec’ tended to bevery homogeneous. The latter is described as a close and homogeneouscommunity that strongly rejected opinions that did not take side with theircause.

‘Strong’ vs. ‘weak’ discussion spaces

Nancy Fraser (1992) defines weak public spheres as ‘publics whose deliberativepractice consists exclusively in opinion formation and does not also encompassdecision making’ and strong public spheres as ‘publics whose discourseencompasses both opinion formation and decision making.’

In order to employ this fundamental distinction for evaluating its impacton the quality of the deliberation, we propose to extrapolate this definition byfocusing on the perceived impact that the participants have of these differentcontexts of discussion.

Davy Janssen and Raphael KiesOnline Forums

323

Acta Politica 2005 40

We define a public space as strong when participants consider that theirparticipation will be read and considered by other users and/or could have someconcrete political outcomes. On the contrary, we classify an online public space asweak when participants do not believe that their participation counts and/or whenthey consider that no concrete political outcome is likely to result from the onlinedebate.

There are different ways for an online public space to be perceivedas strong by its users: (i) It can result from the visibility of the publicspace and therefore its potential political influence (i.e. the number of personsreading the forum); (ii) It can also result from the aim of the forum. Aswe saw, there are an increasing number of web-based discussion spacesinvolving citizens in consultation processes or decision-making pro-cesses; (iii) Finally, it can result from the status and the power of thepeople participating actively or even just passively (just reading) in the onlinedebates.

The literature we came across suggests that strong public spaces tend to bemore deliberative than weak public spaces. Coleman, when comparing theweak governmental forum ‘citizenspace’ with the e-consultation experienceorganized by the Hansard society on stem research, pointed out that in‘citizenspace’ the messages tended to be shorter than within specific e-consultation forums (Coleman et al., 2002).5 This suggests that the level ofjustification tends to be lower in weak public spaces than in strong publicspaces. Similarly, Jensen (2003a) who compared an institutional politicalforum (nordpol.dk) with a widely used Usenet group (dk.politik) observed thatthe messages contained in the institutional forum were not only generallylonger and better justified but also more respectful: they were comparable to a‘letter to the editor’ (Jensen, 2003a, 355). The fact that the forum wasmoderated and that politicians actively participated is considered to haveplayed an important role for explaining the difference in quality ofargumentation. Finally, Monnoyer (2004) — who compared a French onlineforum on environmental issues with an e-consultation on the constructionof a third airport in the region of Paris — observed that participants in thee-consultation public space tended to be more polite, to raise more questionsand to be more precise when they reported discourses of others than participantsin environmental Usenet groups. Monnoyer also observed that there is apersistence of deliberating seriously in the e-consultation forum even whenparticipants expressed many doubts as to whether they were heard.

In summary, the empirical findings we have accumulated so far suggest thefollowing:

� Asynchronous types of forum are more adapted to host political anddeliberative debates than synchronous discussion spaces.

Davy Janssen and Raphael KiesOnline Forums

324

Acta Politica 2005 40

� The request of identification and the guarantee of freedom of expression (nocensure) are important factors to promote the quality and continuity of thepolitical debate.

� The ideologically extreme institutions/communities are less likely to hostopen and plural forms of debates.

� The topic of the debate is a relevant variable for explaining differencesobserved in the level of participation, gender distribution and quality ofdeliberation.

� In discursive contexts where participants think that their voices can have animpact on decisions (strong public spaces) they are more ready and willing tospend time to elaborate and to justify their opinions.

Measuring Deliberation Online

The link between the theory of deliberative democracy and the practice ofonline forums is the subject of an emerging body of literature. The researchquestion concerning the relationship between online forums and theories ofdemocracy and public space is a recurring one. Graham asks ‘to what extent,do current online political forums correspond to the ideal notion of the publicsphere advocated by Habermas and other deliberative democrats?’ (Graham,2002, 9). Schneider tests ‘the hypothesis that the form of discourse fostered bycomputer-mediated discussion provides opportunities to expand the informalzone of the public sphere’ (Schneider, 1997, 1). Wilhelm asks ‘how useful arethese virtual sounding boards in enabling deliberation in the public sphere?’(Wilhelm, 1999, 1), and Jensen echoes ‘can [the internet] contribute tostrengthening democracy by creating new public spheres online?’ (Jensen,2003a, 1).

Each of these authors has developed an instrument to measure the extent towhich online forums have approached the conditions of an idealized publicsphere. In this article, we will present an overview of some of the previousresearch and make suggestions for future research. Our emphasis will lie on theempirical aspect, on the operationalization of public space criteria.

Operationalisation of online deliberation

Public sphere theory and deliberative democratic theory are heavily influencedby the work of Jurgen Habermas (Habermas, 1984/7, 1988, 1996). Habermas’stheory of communicative action and discourse ethics provides a strong basisfor the conceptualization of an ideal type of public sphere. Researchers whohave operationalized this concept for empirical research do not agree on theselection of criteria that constitute the idealized public sphere. Schneider lists

Davy Janssen and Raphael KiesOnline Forums

325

Acta Politica 2005 40

the following ‘four dimensions that embody the spirit of the idealised publicsphere’: equality, diversity, reciprocity and quality (Schneider, 1997, 72). Jensenselects six variables: form, dialogue, openness, tone, argumentation andreciprocity (Jensen, 2003a). Jankowski and Van Os suggest ‘a combination ofthe dimensions suggested by both Schneider and Dahlberg’ (Jankowski andvan Os, 2002, 3). Wilhelm uses a modified list with criteria of the virtual publicsphere, which are topography, topicality, inclusiveness, design and deliberation(Wilhelm, 1999). Steenbergen et al. list the following criteria: participation,level of justification, content of justification, respect and constructive politics(Steenbergen et al., 2003). Finally, Graham distinguishes between the process ofunderstanding (consisting of rational-critical debate, reciprocity, reflexivity andempathy), sincerity, equality and freedom (Graham, 2002).

In this article, we look at the way in which these authors have sought toconstruct measurement schemes for their criteria, each of which covers aspecific aspect of an idealized public sphere. A list of criteria we have notmentioned here, but one we believe is quite exhaustive in that it covers all the‘ingredients’ of the other lists is that of Lincoln Dahlberg (2002). These are hiscriteria: thematization and reasoned critique of problematic validity claims(reciprocity and justification), reflexivity, ideal role taking, sincerity, inclusionand discursive equality, and autonomy from state and economic power. For thesake of clarity, we will structure the rest of the article according to Dahlberg’slist. This is not to say that we prefer it to other efforts at finding public spherecriteria, but it does provide us with a fairly extensive and theoreticallygrounded set of criteria.

Characteristics of the idealized public sphere

ReciprocityThe element of reciprocity captures the degree to which a conversation is a real‘discussion’. Graham defines reciprocity as ‘the taking in (listening, reading)of another’s claim or reason and giving a response’ (Graham, 2002, 45).For Schneider, reciprocity ‘refers to the notion that people are engaged inconversation with each other, and that their messages are reflected upon anddiscussed by others’ (Schneider, 1997, 74). In previous research, reciprocityhas predominantly been operationalized in a content analysis6 that codes forthe number of replies a message gets. Jensen uses the categories of ‘initiate’(a message initiates a new debate), ‘reply’ (message is a reply to a previousmessage) and ‘monologue’ (message is not really part of a debate), whereasGraham uses the comparable categories of ‘initial’, ‘response’ and ‘irrelevant’.7

In each of these cases, messages posted are qualitatively interpreted accordingto these criteria. Schneider employs a purely structural (quantitative) analysis.His is also a ‘reply-count’, but the criteria for reply are purely structural: ‘a

Davy Janssen and Raphael KiesOnline Forums

326

Acta Politica 2005 40

message is considered reciprocal to a previous message if it appears in the samethread within seven days of the previous message, or if it cites the messagedirectly by message identification number’ (Schneider, 1997, 74).

JustificationJustification is a crucial requirement of true deliberation, because ‘if validreasons are not advanced, then subjects may not be able to find commonground’ (Graham, 2002, 43). The evaluation of justification is therefore acrucial element of deliberative quality. Previous research has used variouscontent analysis approaches centred on the arguments that participants useto backup what they are saying. The most crude approach only codes for theabsence or presence of arguments (Wilhelm, 1999). A more elaborate coding,that of Jensen, (2003a), makes a further distinction: when arguments arepresent they are either ‘internal’(based on personal viewpoints and values) or‘external’(based on facts and figures). There is a normativity present in thesecodings in that arguments are better than no arguments and arguments basedon ‘objective’ information are better than those based on personal experience.Another approach to justification operationalizes the Rawlsian construct ofpublic reason and Habermas’s emphasis on the common good by focusingon the ‘scope’ of arguments (for who are they acceptable?). Steenbergen et al.distinguish, in their ‘content of justification’ category, between neutralstatements, statements concerning group interests and statements referring tothe common good.8 A final approach to justification consists of a proceduralor structural evaluation that abstracts the content of argumentation.Steenbergen et al. (2003) ‘level of justification’ category does this by lookingat the completeness of inferences.

ReflexivityIf the defining feature of deliberative democracy is preference change throughdeliberation (Dryzek, 2000), reflexivity is one of its central characteristics.Dahlberg defines reflexivity as follows: ‘participants critically examine theirvalues, assumptions, and interests, as well as the larger social context’(Dahlberg, 2002, 3). Graham talks of ‘the rethinking of one’s own validityclaims and arguments in light of another’s validity claim and/or argument’(Graham, 2002, 46), and according to Jensen research on reflexivity shouldtry to ‘gain a reasonable interpretation of the extent interactions encourageself-critique and position alterations’ (Jensen, 2003a, 361). Whereas neitherSchneider nor Wilhelm deal with the issue in their content analysis, Jensenmeasures reciprocity in a rather crude category with the variables ‘persuasion’,‘progress’ and ‘radicalisation’ (Jensen, 2003a). Graham’s approach is lessbased on a general appreciation of the content of a message and uses an

Davy Janssen and Raphael KiesOnline Forums

327

Acta Politica 2005 40

argumentation analysis approach based on counterarguments and rebuttalsthat, combined with ‘counter evidence’, indicate ‘that participants have movedbeyond their own reasons and justifications and considered the claims andarguments of others’ (Graham, 2002, 67).

The problem with the text analysis approaches described here is that theyleave important aspects of reflexivity untouched. In his Net-public spherearticle, Dahlberg makes the fundamental observation that it ‘is quite difficultto evaluate (reflexivity) because it is a largely internalized process’ (Dahlberg,2002, 6) taking place in the minds of individuals. This is the certainly the casefor a content analysis approach because ‘written communications may onlyshow traces of such a subjective process’ (Dahlberg, 2002, 6). A way out here isan extension of methodology to include participant observations in the formof (online) survey research. Jensen used the survey approach and asked forumparticipants about several ‘internal effects’ on their thinking about the topicsof discussion (Jensen, 2003b). Further survey research on reflexivity in onlineforums has been done by Muhlberger (2000, 2003).

Ideal role takingDahlberg defines his third criterium as ‘ongoing, respectful listening’(Dahlberg, 2002). We first discuss the aspect of respectful listening. Ideal roletaking implies that all positions, ‘not just those immediately present in theforum but all affected by the question under consideration’ (Dahlberg, 2002, 7)are taken into account. Respectful listening moves us into the realm of ‘respect’and its opposite, disrespect, on which some of the previous research has alreadyfocused. Some studies have applied content analysis to look for explicitinstances where respect is absent (Jankowski and Van Os, 2002; Jensen, 2003a).It is assumed that lesser instances of disrespect imply an increase indeliberativeness. Steenbergen et al. code for respect rather elaborately, with acategory for respect for groups, respect towards the demands of others andrespect towards the counterarguments of others.

With this focus on disrespect, the construct of respectful listening, let alonethat of ideal role taking, has not been researched in its entirety. For a moregeneral appreciation, Dahlberg suggests that we look for ‘participants seekingto understand the other through reflecting positions, asking for clarification,acknowledging the rights of all to be heard, and even putting forward positionsthat are not one’s own to enable broader reflection’ (Dahlberg, 2002, 7). Webelieve that this very progressive level of understanding can only be gauged atwith a qualitative approach and an in-depth reading of messages.

The second core element of ideal role taking is its ongoing character.Research has focused on the structure of discussion threads (Schneider, 1997;Wilhelm, 1999). In his content analysis, Wilhelm codes for ‘time’ (the mean

Davy Janssen and Raphael KiesOnline Forums

328

Acta Politica 2005 40

time length of a thread in days) and ‘thread’ (the mean number of threadsper day) (Wilhelm, 1999). The thread is taken as a carrier of conversation andthe amount and duration of threads are the criteria for the ongoing characterof discussions. Another measurement that has been used in previous researchconsists of the counting of the number of participants that have onlyparticipated in the forum one time (that have only sent one message), referredto as the ‘one-timer effect’ (Graham, 2002). The rationale of this measurementis that conversations will not be ongoing if a lot of the participants just ‘say’something one time and then leave.

SincerityThe requirement of sincerity implies that ‘participants must make a sincereeffort to make known all relevant information including their intentions,interests, needs, and desires’ (Dahlberg, 2002, 3). Communicative actionrequires sincerity and banishes rhetorical forms of speech to the realm ofstrategic action. Textual analysis approaches, looking for sincerity in onlineconversations, have mostly focused on traces of the absence of sincerity.Graham approaches insincerity in a qualitative manner, looking for instanceswhere forum participants accuse other participants of being insincere. Heacknowledges that he is thus in fact measuring insincerity as perceived by otherparticipants but argues ‘it is the perception of sincerity, which has the greatestimpact on the process of deliberation’ (Graham, 2002, 70). Dahlberg suggestsa more comprehensive approach with a qualitative analysis based on‘consistency in speech, consistency in speech and action, and coherence’(Chambers 1996, cited in Dahlberg 2002, 7). An inductive approach could lookfor instances where participant’s inconsistencies are exposed by otherparticipants or could look for inconstencies in speech and/or action themselves.

Inclusion and discursive equalityThe requirement of inclusion implies that all who are affected by the issuesunder discussion, or more generally all who are interested, should be able toparticipate. In an online environment this obviously means access to acomputer with an internet connection and the necessary ICT skills to reach anonline forum and make a contribution. Research can report descriptivestatistics concerning internet penetration, PC ownership, etc., and can presentsurvey data on internet use and ICT skills. Furthermore, the moderationregime and/or technical architecture of an online forum can make access easieror harder and can thus be reported as a variable impacting on inclusion.

Another issue, that of discursive equality, concerns us more. It concernsinclusion among those that are able to access a forum, where ‘each participant(should have) an equal opportunity to introduce and question any assertion

Davy Janssen and Raphael KiesOnline Forums

329

Acta Politica 2005 40

whatsoever and to express attitudes, desires, and needs’ (Dahlberg, 2002, 3).Graham states that previous studies have dealt with discursive equality ‘fromtwo slightly different perspectives: equal voice and equal standing’ (Graham,2002, 70). The ‘equal standing’ research has been mostly qualitative researchthat analysed the discourses used by different groups of participants (men andwomen, professionals and dilettants) to evaluate if certain discourses aredominating (and thus distracting from equality). We have not had the time todeal with this strand of research as of yet, and will thus reserve any commentsfor the future. The ‘equal voice’ perspective has gained a lot of attention inprevious research (Schneider, 1997; Graham, 2002; Jensen, 2003a). It analyses‘distribution of voice’ in a conversation in the assumption that if only a smallamount of participants makes a large proportion of the contributions they thendominate this debate. This idea of domation of conversation is operationalizedin a ‘%participant–%contributions’ statistic: the distribution of the amount ofmessages posted per participant is plotted on a Lorenz curve, often revealingthat a small percentage of posters is responsible for a large percentage ofmessages posted and thus indicating the domination of the conversation by onegroup of participants.

Autonomy from state and economic powerThis final but fundamental criteria of the idealized public sphere requires that‘deliberation is driven by the concerns of publicly-oriented citizens ratherthan by money or administrative power’. It concerns the ‘place’ where theconversation takes place and is derived from Habermas’s lifeworld/systemstheory (Habermas, 1984/1987). It is communicative rationality that shoulddrive deliberation in online forums, and not the ‘instrumental’ rationality ofthe systems world. Concretely, online deliberation should be free fromintrusion by state and economic power. The empirical studies considered herehave mostly not dealt with this criterium explicitly, except for some that ask thequestion of who is allowed to set the agenda of the conversations. Dahlbergproposes some descriptive reporting on explicit instances where there is clearlyno autonomy (presence of censorship or other forms of surveillance, on-siteadvertisements, etc.) (Dahlberg, 2002). However, a deeper insight into thesubtle ways in which the steering mechanisms of money and power can curbautonomy, according to Dahlberg, can only be gained through participantobservations.

Conclusion

In the first section of the article, a selection of empirical studies on onlinedeliberation and their findings have been discussed. We gathered many more

Davy Janssen and Raphael KiesOnline Forums

330

Acta Politica 2005 40

studies though, and could have focused the entire article on research findings.The reason why we did not emerges already from the presentation of selectedfindings: no conclusive statements on the deliberative quality of online forumscan be made. Findings differ enormously, sometimes pointing at traces oftrue deliberation, often pointing at its absence. That is why we focused onmethodological issues in the bulk of our article: to see if different findings canbe explained by differences in methodology.

We found that the empirical research on deliberation in online forums hasmostly taken the form of a ‘discrepancy analysis’ in which a set of criteria forthe idealized public sphere is derived from a theoretical construct, and in whichonline conversations are analysed (in a textual analysis) to see to which degreethey approach the criteria of the ideal type. Several criticisms can be madeconcerning this research. First, the often considerable differences between thechoice of criteria for deliberation (and the way they are then operationalized)are problematic, not in the least from the perspective of comparability ofresearch findings. Some studies omit fundamental criteria, for exampleSchneider’s (1997) that does not deal with justification (and thus ignores thequality of argumentation). Second, we argue that a purely structural/procedural analysis cannot result in meaningful findings on deliberative qualityand needs to be complemented by a qualitative analysis.9 Certainly for‘justification’ a qualitative approach is needed to be able to interpret the kindsof arguments that are put forward and to gain insight into what kind ofargumentation is used, (not) appreciated and (not) tolerated in the forum.

Third, research that aims at a comprehensive account of the democraticimportance of online forums, we argue, should include participant observation,for example in the form of online surveys or sociological interviews. Theexistence of ‘lurkers’10 by itself is enough reason to also include surveys onperceived reciprocity, reflexivity, equality and justification. Otherwise, thisgroup of participants can never be reached. Furthermore, the combination ofa survey approach and a textual analysis based on a sampling strategy thatresults in a comprehensive selection of conversation threads, could result incomplementary observations that will strengthen research findings. Fourth, thedeductive approach of past research, whereby a set of criteria derived from atheory is applied to a conversation practice, can be problematic in itself. Forexample, to what degree can we apply Habermasian criteria to an online forumthat is purely discursive, not tied to any decision making and not part ofinstitutional politics? In this type of forum people discuss politics but are notlooking for agreement or consensus, whereas in the Habermasian frameworkconversations are directed at some outcome, agreement or consensus.Furthermore, the Habermasian criteria favour a certain type of language thatis ‘abstract (y), ethically acceptable, presented in a cooperative manner andwith transparent intentions, and makes a claim to truth’ (Markovits, 2004, 19).

Davy Janssen and Raphael KiesOnline Forums

331

Acta Politica 2005 40

Authors such as Lynn Sanders and Iris Young have pointed out that in thepublic space people should be able to narrate in their own terms (Dryzek,2000). These and other ‘difference democrats’ plead for the inclusion of otherforms of communication, such as testimony, storytelling, rhetoric and greeting(Dryzek, 2000). A predetermined set of Habermasian criteria downplays therelevance of those language forms. Especially in online forums where ordinarycitizens are invited this seems an unjust approach.

It seems to us that all the textual analysis approaches discussed are flawedin a similar manner. The constructs which they seek to operationalize arecomplex, so that reductions of this complexity in the form of mutuallyexclusive content analysis coding categories will always lead to a loss ofmeaning. The fundamental consideration to be made is whether this reductionof complexity still allows for meaningful research findings. We argue that forsome of the empirical research discussed here the answer is no. The strength ofthe content analysis approach, especially that of the quantitative variant, is itsability to analyse large amounts of messages. Content analysis is an establishedmethod of the social sciences and carries with it implications of controllabilityand representativeness. The nature of the research question (how to get a trueidea of the deliberative quality of an online conversation), however, begs fora more qualitative approach that enables the researcher to reveal discoursepatterns in a more inductive manner, instead of a top-down application ofsome content analysis template.

In any case, the methodological debate is ongoing and researchers arecoming together to compare their approaches to allow for true cross-comparative research on online deliberation.11 In our article, we outlinedseveral variables that could be relevant in explaining differences in deliberativequality. It is only when researchers will agree on certain methodologicalapproaches that the relevance of the communicative structure of discussionspaces, the political culture and ideology, and the strong vs. weak distinctionfor explaining differences in deliberative quality will become clear.

References

Aikens, G. (1997) ‘American democracy and computer-mediated communication: a case study in

Minnesota’, Ph.D dissertation, Department of Social and Political Sciences, Cambridge

University.

Beierle, T.C. (2002) ‘Democracy online: an evaluation of the National Dialogue on Public

Involvement in EPA Decision’, RFF Report, Washington.

Bentivegna, S. (1998) Talking politics on the net, Research paper R-20, Cambridge, MA: School of

Government, JFK, Harvard University.

Chambers, S. (1996) Reasonable Democracy: Jurgen Habermas and the Politics of Discourse,

London: Cornell University Press.

Coleman, S., Hall, N. and Howell, M. (2002) Hearing Voices. The Experience of Online Public

Consultation and Discussion in UK Governance, UK: Hansard Society.

Davy Janssen and Raphael KiesOnline Forums

332

Acta Politica 2005 40

Dahlberg, L. (2002) ‘Net-public sphere research: beyond the ‘first phase’, Euricom Colloquium:

Electronic Networks and Democracy; 9–12 October 2002; Nijmegen, The Netherlands.

Davis, R. (1999) The Web of Politics, Oxford: Oxford University Press.

Docter, S. and Dutton, W.H. (1998) ‘The First Amendment Online: Santa Monica’s Public

Electronic network, in R. Tsagarousianou, D. Tambini and C. Bryan (eds.) Cyberdemocracy:

Technologies, Cities and Civic Network, London: Routledge.

Dryzek, J.S. (2000) Deliberative Democracy and Beyond, Oxford: Oxford University Press.

Dumoulin, M. (2003) ‘Les Forums Electroniques: Deliberatifs et Democratiques?’, in D. Moniere

(ed.) Internet et la Democratie, Montreal: Erudit.

Dutton, W.H. (1996) ‘Networks rules of order: regulating speech in public electronic fora’, Media,

Culture & Society 18: 269–290.

Fraser, N. (1992) ‘Rethinking the Public Sphere: A Contribution to the Critique of Actually

Existing Democracy’, in C. Calhoun (ed.) Habermas and the Public Sphere, Cambridge: MIT

Press.

Graham, T.S. (2002) ‘The public sphere needs you. Deliberating in online forums: new hope for the

public sphere?’, Master Thesis, University of Amsterdam.

Habermas, J. (1984/1987) The Theory of Communicative Action: Volumes 1&2, Boston: Beacon

Press.

Habermas, J. (1988) The Structural Transformation of the Public Sphere, Cambridge: MIT Press.

Habermas, J. (1996) Between Facts and Norms: Contributions to a Discourse Theory of Law and

Democracy, Cambridge: MIT Press.

Iyengar, S., Luskin, R.C. and Fishkin, J.S. (2004) ‘Facilitating informed political opinion:

evidences from face-to-face and online deliberative polls’, Research Paper, The Center for

Deliberative Democracy, Stanford University, Stanford.

Jankowski, N. and van Selm, M. (2000) ‘The Promise and Practice of Public Debate in cyberspace’,

in K. Hacker and J.A.G.M. Van Dijk (eds.) Digital Democracy: Issues of Theory and Practice,

London: Sage.

Jankowski, N.W. and van Os, R. (2002) ‘Internet-based political discourse: a case study of

electronic democracy in the city of Hoogeveen’, Euricom Colloquium: Electronic Networks and

Democracy; 9–12 October 2002; Nijmegen, The Netherlands.

Jensen, J.L. (2003a) ‘Public spheres on the internet: anarchic or government-sponsored — a

comparison’, Scandinavian Political Studies 26(4): 349–374.

Jensen, J.L. (2003b) ‘Minnesota E-democracy — online participation and political effects’,

Association of Internet Researcher’s Conference; 16–19 October 2003; Toronto, Canada.

Kies, R. (2003) ‘E-democracy strategy of Partito Radicale’, e-working paper, E-Democracy Center,

Geneva: University of Geneva: http://edc.unige.ch/publications/edcreports/STOA/casestudies/

partito_radicale.pdf.

Maldonado, T. (1997) Critica della Ragione Informatica, Milano: Feltrinelli.

Markovits, E. (2004) ‘The trouble with being earnest: deliberative democracy and the sincerity

norm’, Annual Meeting of the American Political Science Association; 2–5 September; Chicago,

USA.

Monnoyer-Smith, L. (2004) ‘Nouveaux dispositifs de concertation et formes controverses: le choix

d’un troisieme aeroport pour Paris’, Rapport intermediaire interne, Laboratoire Communica-

tion et Politique, CNRS, Paris.

Muhlberger, P. (2000) ‘Defining and measuring deliberative participation and potential: a

theoretical analysis and operationalization’, International Society for Political Psychology 23rd

Annual Scientific Meeting; Seattle, USA.

Muhlberger, P. (2003) ‘Political values, political attitudes, and attitude polarization in

internet political discussion: political transformation or politics as usual’, Communications 28:

107–133.

Davy Janssen and Raphael KiesOnline Forums

333

Acta Politica 2005 40

Schneider, S.M. (1997) ‘Expanding the public sphere through computer-mediated communication:

political discussion about abortion in a Usenet newsgroup’, Ph.D. dissertation, Department of

Political Sciences, Massachusetts Institute of Technology.

Schultz, T. (2000) ‘Mass media and the concept of interactivity: an exploratory study of online and

reader email’, Media, Culture and Society 22: 205–221.

Steenbergen, M.R. et al. (2003) ‘Measuring political deliberation: a discourse quality index’,

Comparative European Politics 1(1): 21–48.

Sunstein, C. (2001) Republic.com, New Jersey: Princeton University Press.

Tanner, E. (2001) ‘Chilean conversations: internet forum participation debate Augusto Pinochet’s

detention’, Journal of Communication 51: 383–402.

Trechsel, A., Kies, R., Mendez, F. and Schmitter, P. (2004) ‘Evaluation of the use of new

technologies in order to facilitate democracy in Europe: E-democratizing the parliaments and

parties in Europe’, European Parliament DG for research: STOA, http://edc.unige.ch/

publications/edcreports/STOA/main_report.pdf.

Wilhelm, A.G. (1999) ‘Virtual sounding boards: how deliberative is online political discussion?’, in

B.N. Hague and B.D. Loader (eds.) Digital Democracy. Discourse and Decision Making in the

Information Age, London: Routledge.

Notes

1 For information we mention two interesting examples of online deliberative experiments: the

Seniorweb in the Netherlands that aimed to allow the senior citizens to get politically involved

(Jankowski and van Selm, 2000) and the online deliberative opinion poll, which corresponds to a

tentative of adapting the famous experiences offline deliberative opinion poll experiences to the

online environment (Iyengar et al., 2004).

2 See www.radicali.it. Its forum has existed for more than 4 years and contains more than 360,000

interventions with more than 23,000 people that are registered.

3 As the treasurer of the party, Rita Bernardini says in an interview: ‘if the forum was anonymous

I would not bother to reply to provocation, but since the forum is public with forename and

name, I feel that I have to reply.’ (interview with Rita Bernardini, 2004).

4 The current responsible of the forum, noticed that once the moderation was eliminated

everything went much better: ‘from that moment I made up my mind that the less you moderate,

the less you control these things, the more participants are encouraged to control themselves.’

(Interview with Antonio Grippo, 2004).

5 According to his findings the former had an average of 79 words per message while the latter

had an average of 345 words per message.

6 The unit of coding, for most authors, is the individual message that is posted to the forum. An

exception is Steenbergen et al. (2003) whose unit of coding is a ‘demand’, defined as: ‘a proposal

on what decision should or should not be made’.

7 Wilhelm’s approach is somewhat different because of his category of ‘incorp’ (‘ideas drawn from

others but not from other participants’), but his second category is again ‘reply’.

8 For this last category a further distinction is made between statements of the common good in

utilitarian terms and in terms of the difference principle.

9 We just cannot get around the intuitive feeling that in order to evaluate a message’s deliberative

quality one should actually read them (and thus not only look at their structural or formal

characteristics).

10 ‘Lurkers’ are those forum participants that do not post messages but only read them. Most

forums have a ‘reply-count’ as well as a ‘view-count’ built into their technical architecture that

Davy Janssen and Raphael KiesOnline Forums

334

Acta Politica 2005 40

allows researchers to see how many persons only ‘viewed’ the message (but didnot respond).

Mostly, the group of lurkers will be bigger than the active participants, and often they are more

than two-third of the population.

11 Consider, for example, the ASEF Workshop on Internet Research Methods, 3–5 July, Chinese

University of Hong Kong, with special attention to methodological issues related to

comparative and cross-national studies.

Davy Janssen and Raphael KiesOnline Forums

335

Acta Politica 2005 40

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.