Noname manuscript No.(will be inserted by the editor)

Judgment Aggregation in Multi-AgentArgumentation

Edmond Awad · Richard Booth ·Fernando Tohme · Iyad Rahwan

Received: date / Accepted: date

Abstract Given a set of conflicting arguments, there can exist multiple plau-sible opinions about which arguments should be accepted, rejected, or deemedundecided. We study the problem of how multiple such judgments can beaggregated. We define the problem by adapting various classical social-choice-theoretic properties for the argumentation domain. We show that while argument-wise plurality voting satisfies many properties, it fails to guarantee the collec-tive rationality of the outcome, and struggles with ties. We then present moregeneral results, proving multiple impossibility results on the existence of anygood aggregation operator. After characterising the sufficient and necessaryconditions for satisfying collective rationality, we study whether restricting thedomain of argument-wise plurality voting to classical semantics allows us toescape the impossibility result. We close by listing graph-theoretic restrictionsunder which argument-wise plurality rule does produce collectively rationaloutcomes. In addition to identifying fundamental barriers to collective argu-ment evaluation, our results open up the door for a new research agenda forthe argumentation and computational social choice communities.

Keywords Argumentation · Agents · Preferences · Social Choice

Edmond AwadMasdar Institute of Science and Technology, UAEE-mail: eawad@masdar.ac.ae

Richard BoothUniversity of Luxembourg, LuxembourgE-mail: ribooth@gmail.com

Fernando TohmeArtificial Intelligence Research and Development Lab (LIDIA), Universidad Nacional delSur, Bahıa Blanca, CONICET, ArgentinaE-mail: ftohme@criba.edu.ar

Iyad RahwanMasdar Institute of Science and Technology, UAEUniversity of Edinburgh, UKE-mail: irahwan@acm.org

arX

iv:1

405.

6509

v1 [

cs.A

I] 2

6 M

ay 2

014

2 Edmond Awad et al.

1 Introduction

Argumentation1 has recently become one of the key approaches to automatedreasoning and rational interaction in Artificial Intelligence [5,27]. A key mile-stone in the development of argumentation in AI has been Dung’s landmarkframework [15]. Arguments are viewed as abstract entities (a set A), with abinary defeat relation (denoted ⇀) over them. The defeat relation capturesthe fact that one argument somehow attacks or undermines another. This viewof argumentation enables high-level analysis while abstracting away from theinternal structure of individual arguments. In Dung’s approach, given a set ofarguments and a defeat relation, a rule specifies which arguments should beaccepted.

Often, there are multiple reasonable ways in which an agent may evaluatea given argument structure (e.g. accepting only conflict-free, self-defendingsets of arguments). Each possible evaluation corresponds to a so-called exten-sion [15] or labelling [8,9]. Different argumentation semantics yield differentrestrictions on the possible extensions. Most previous research has focused onevaluating and comparing different semantics based on the (objective) logicalproperties of their extensions [3].

At the heart of most established argumentation semantics is the conditionof admissibility : that accepted arguments must not attack one another, andmust defend themselves against counter-arguments, by attacking them back.Another more restricted notion is called completeness, and is captured, interms of labeling, in the following two conditions:

1. An argument is labelled accepted (or in) if and only if all its defeaters arerejected (or out).

2. An argument is labelled rejected (or out) if and only if at least one of itsdefeaters is accepted (or in).

Otherwise, an argument may be labelled undec. Thus, evaluating a set ofarguments amounts to labelling each argument using a labelling functionL : A → {in, out, undec} to capture these three possible labels. Any la-belling that satisfies the above conditions is also called a legal labelling. Wewill often use legal labelling and complete labelling interchangeably.

The above conditions attempt to evaluate arguments from a single point ofview. Indeed, most research on formal models of argumentation discounts thefact that argumentation takes place among self-interested agents, who mayhave conflicting opinions and preferences over which arguments end up beingaccepted, rejected, or undecided. Consider the following simple example.

Example 1 (A Murder Case) A murder case is under investigation. To startwith, there is an argument that the suspect should be presumed innocent (a3).However, there is evidence that he may have been at the crime scene at thetime (a2), which would counter the initial presumption of innocence. Thereis also, however, evidence that the suspect was attending a party that day

1 This paper is a substantially extended and revised version of [28].

Judgment Aggregation in Multi-Agent Argumentation 3

(a1). Clearly, a1 and a2 are mutually defeating arguments since the suspectcan only be in one place at any given time. Hence, we have a set of arguments{a1, a2, a3} and a defeat relation ⇀= {(a1, a2), (a2, a1), (a2, a3)}. There arethree possible labellings that satisfy the above conditions:

– L(a1) = in, L(a2) = out, L(a3) = in.– L′(a1) = out, L′(a2) = in, L′(a3) = out.– L′′(a1) = undec, L′′(a2) = undec, L′′(a3) = undec.

The graph and possible labellings are depicted in Figure 1.

a1 a2L a3

a1 a2L’ a3

a1: There is credible evidence

that the suspect was at a party.

a2: A witness saw someone

dressed like the suspect at

the crime scene.

a3: The suspect is presumed

innocent.

a1 a2L’’ a3in out undec

Fig. 1 Argument graph with three possible labellings

Example 1 highlights a situation in which multiple points of view can betaken, depending on whether one decides to accept the argument that thesuspect was at the party or the crime scene. The question we explore in thispaper can be highlighted through the following example, extending Example1.

Example 2 (Three Detectives) A team of three detectives, named 1, 2, and 3,have been assigned to the murder case described in Example 1. Each detective’sjudgment can only correspond to a legal labelling (otherwise, her judgment canbe discarded). Suppose that each detective’s judgment is such that L1 = L,L2 = L′ and L3 = L′. That is, detectives 2 and 3 agree but differ withdetective 1. These labellings are depicted in the labelled graph of Figure 2.The detectives must decide which (aggregated) argument labelling best reflectstheir collective judgment.

Example 2 highlights an aggregation problem, similar to the problem ofpreference aggregation [2,16,32] and the problem of judgment aggregation onpropositional formulae [23,20,22,19]. It is perhaps obvious in this particu-lar example that a3 must be rejected (and thus the defendant be consideredguilty), since most detectives seem to think so. For the same reason, a1 mustbe rejected and a2 must be accepted. Thus, labelling L′ (see Example 1) winsby majority. As we shall see in our analysis below, things are not that simple,and counter-intuitive situations may arise. We summarise the main questionasked in the paper as follows.

4 Edmond Awad et al.

L1 =

in out

a1 a2 a3

1

L2 =

2

a1 a2 a3

L3 =

3

a1 a2 a3

Collective

labelling?

a1: There is credible evidence

that the suspect was at a party.

a2: A witness saw someone

dressed like the suspect at

the crime scene.

a3: The suspect is presumed

innocent.

Fig. 2 Three detectives with different judgments

Given a set of agents, each with a specific subjective evaluation (i.e.labelling) of a given set of conflicting arguments, how can agents reacha collective decision on how to evaluate those arguments?

While Arrow’s Impossibility Theorem can be expected to ensue for thisproblem [1]2, there exist many differences between labellings and preferencerelations (for which Arrow’s result apply), stemming from their correspond-ing order-theoretic characterisations. In other words, aggregating preferencesassumes that agents submit a full order of preferences over candidates, whilein labelling aggregation, agents submit their top labeling for a set of logicallyconnected arguments.

The problem of labelling aggregation is more comparable to the judgementaggregation problem [23,20,22,19], by considering arguments as propositionswhich are logically connected by the conditions of legal labelling. However,one important difference is that in JA, each proposition can have two values:True or False. In labelling aggregation, on the other hand, each argument canhave three values: in, out, or undec. This makes labelling aggregation be morecomparable to 3-value JA [13,14].

The 3-value JA problem was proposed by Dokow and Holzman in [13,14].However, labelling aggregation is still different from 3-value JA. The maindifference is the type of logical connections between arguments/propositions.Argumentation graphs are subject to specific constraints that characterise ac-ceptable labels, and many semantics have been proposed to deal with ac-ceptability of arguments, while 3-value JA only deals with one acceptabilitysemantics, namely propositional satisfiability. Additionally, argumentation es-sentially abstracts contradictory knowledge bases which is not the case with3-value JA.

In this paper, we conduct an extensive social-choice-theoretic analysis ofargument evaluation semantics by means of labellings. We start by studyingthe problem of aggregating different individual judgments on how a set of argu-ments should be evaluated. We introduce the argument-wise plurality rule andstudy its key properties. We also present key impossibility results on the exis-tence of a good aggregation operators. We then fully characterise the sufficient

2 Arrow’s Theorem claims that four quite natural constraints, that capture abstractly theproperties of a democratic aggregation process, cannot be simultaneously satisfied.

Judgment Aggregation in Multi-Agent Argumentation 5

and necessary conditions under which Collective Rationality is guaranteed forany aggregation operator. We analyse argument-wise plurality rule with re-spect to this characterisation and we study whether the restriction of thedomain to classical semantics would ensure the fulfillment of these conditions.Finally, we provide graph theoretical restrictions under which argument-wiseplurality rule produces collectively rational outcomes.

The paper makes three distinct contributions to the state-of-the-art in thecomputational modelling of argumentation. Firstly, the paper introduces thestudy of aggregating different individual judgments on how a given set of argu-ments is to be evaluated. This requires adapting classical social-choice proper-ties to the argumentation domain, and sometimes demands special treatment(e.g. different versions of some properties). The paper highlights that aggre-gating different argument evaluations can lead to similar paradoxes to thosestudied in other areas of social choice theory (e.g. the well-known CondorcetParadox in preference aggregation, and the Discursive Dilemma in the judg-ment aggregation literature).

The second contribution of this paper is proving the impossibility of theexistence of any aggregation operator that satisfies some minimal properties.In doing so, we show impossibility results that concern dealing with ties andproducing a collectively rational evaluation of arguments. Moreover, we pro-vide a full characterisation of the necessary and sufficient conditions underwhich an aggregation operator is guaranteed to produce a rational evaluationof arguments. The impossibility results that we present build on the solid foun-dation of social choice and judgement aggregation for impossibility results suchas Arrow’s celebrated impossibility theorem [1], Sen’s impossibility theorem[30], the Muller–Satterthwaite theorem [24], the Gibbard-Satterthwaite theo-rem [17,29], and List and Pettit’s impossibility result on judgment aggregationin propositional inference [21]. Hence, as is the case with other aggregation do-mains, the aggregation paradox in argument evaluation is an example of a morefundamental barrier. These results are important because they give conclusiveanswers and focus research in more constructive directions (e.g. weakening thedesired properties in order to avoid the paradox).

The third contribution of this paper is an extensive analysis of an aggre-gation rule, namely argument-wise plurality rule. We analyse the properties ofargument-wise plurality rule in general, and investigate whether the restric-tion of the domain of votes to a particular classical semantics would ensure thefulfillment of these conditions. This highlights a novel use of classical seman-tics, which are originally used to resolve issues in single-agent nonmonotonicreasoning. Finally, we provide graph theoretical restrictions on argumentationframeworks under which argument-wise plurality rule would be guaranteed toproduce collectively rational outcomes.

The paper is organised as follows. In section 2, we start by giving a briefbackground on abstract argumentation systems. Sections 3, 4, 6 and 7 focuson the problem of aggregating sets of judgments over argument evaluation.Sections 5, 8, and 9 focus on introducing and analysing argument-wise pluralityrule. We conclude the paper and discuss some related work in Section 10.

6 Edmond Awad et al.

2 Background

In this section, we briefly outline key elements of abstract argumentationframeworks. We begin with Dung’s abstract characterisation of an argumen-tation system [15]. We restrict ourselves to finite sets of arguments.

Definition 1 (Argumentation framework) An argumentation frameworkis a pair AF = 〈A,⇀〉 where A is a finite set of arguments and ⇀⊆ A×A is adefeat relation. We say that an argument a defeats an argument b if (a, b) ∈⇀(sometimes written a ⇀ b).

a3 a2

a4

a1

a5

Fig. 3 A simple argument graph

An argumentation framework can be represented as a directed graph inwhich vertices are arguments and directed arcs characterise defeat among ar-guments. An example argument graph is shown in Figure 3. Argument a1has two defeaters (i.e. counter-arguments) a2 and a4, which are themselvesdefeated by arguments a3 and a5 respectively.

There are two forms of semantics that define the decision to make aboutthe arguments. One of them is extension-based semantics by Dung [15], whichproduces a set of arguments that are accepted together. Another equivalentlabelling-based semantics is proposed by Caminada [8,9], which gives a la-belling for each argument. With argument labellings, we can accept arguments(by labelling them as in), reject arguments (by labelling them as out), andabstain from deciding whether to accept or reject (by labelling them as undec).Caminada [8,9] established a correspondence between properties of labellingsand the different extensions. In this paper, we employ the labelling approach.

Definition 2 (Argument Labelling) Let AF = 〈A,⇀〉 be an argumenta-tion framework. An argument labelling is a total function L : A → {in, out,undec}.

We will make use of the following notation.

Definition 3 Let AF = 〈A,⇀〉 be an argumentation framework, and L alabelling over AF . We define:

– in(L) = {a ∈ A | L(a) = in}– out(L) = {a ∈ A | L(a) = out}– undec(L) = {a ∈ A | L(a) = undec}

A labelling L can be represented as L = (in(L), out(L), undec(L)).

Judgment Aggregation in Multi-Agent Argumentation 7

For example, in Figure 4, we have LG = ({a3}, {}, {a1, a2}), L1 = ({a1, a3}, {a2}, {}),and L2 = ({a2, a3}, {a1}, {}).

In the rest of the paper, by slight abuse of notation, when we refer to a la-belling L as an extension, we will be referring to the set of accepted argumentsin(L).

Labellings must satisfy the condition that an argument is in if and only ifall of its defeaters are out. An argument is out if and only if at least one ofits defeaters is in. This requirement is captured in the definition of completelabelling.

Definition 4 (Complete Labelling) Let AF = 〈A,⇀〉 be an argumenta-tion framework. A complete labelling is a total function L : A → {in, out,undec} such that:

– ∀a ∈ A : (L(a) = out ≡ ∃b ∈ A such that (b ⇀ a and L(b) = in)); and– ∀a ∈ A : (L(a) = in ≡ ∀b ∈ A : ( if b ⇀ a then L(b) = out))

Otherwise, L(a) = undec (since L is a total function). We will use Comp(AF )to denote the set of all complete labellings for AF .

As an example, consider the following.

Example 3 Consider the graph in Figure 4. Here, we have three completelabellings: LG = ({a3}, {}, {a1, a2}), L1 = ({a1, a3}, {a2}, {}), and L2 =({a2, a3}, {a1}, {}).

a2 a1

a3

in out

L1

a2 a1

a3L2

a2 a1

a3LG

undec

Fig. 4 Graph with three complete labellings.

In addition to the complete labelling, there are other semantics which as-sume further conditions.

Definition 5 (Other Labellings) Let AF = 〈A,⇀〉 be an argumentationframework. Let L : A → {in, out, undec} be a complete labelling.

– L is a grounded labelling if and only if in(L) is minimal, or equivalentlyout(L) is minimal, or equivalently undec(L) is maximal (w.r.t set inclu-sion).

– L is a preferred labelling if and only if in(L) is maximal, or equivalentlyout(L) is maximal (w.r.t set inclusion).

– L is a semi-stable labelling if and only if undec(L) is minimal (w.r.t setinclusion).

8 Edmond Awad et al.

– L is a stable labelling if and only if undec(L) = φ.

Consider the following example.

Example 4 Consider the graph in Figure 4. Here, we have the grounded la-belling is LG = ({a3}, {}, {a1, a2}). We have only two preferred labellings:L1 = ({a1, a3}, {a2}, {}), and L2 = ({a2, a3}, {a1}, {}). These are also theonly stable and semi-stable labellings for this framework.

We refer to the previous semantics as classical semantics. There exist othersemantics which we do not consider in this work.

3 Aggregation of Argument Labellings

To date, most analyses inspired by Dung’s framework have focused on analysingand comparing the properties of various types of extensions/labellings (i.e. se-mantics) [3]. The question is, therefore, whether a particular type of labellingis appropriate for a particular type of reasoning task in the presence of con-flicting arguments.

In contrast with most existing work on Dung frameworks, our concernhere is with multi-agent systems. Since each labelling captures a particularrational point of view, we ask the following question: Given an argumentationframework and a set of agents, each with a legitimate subjective evaluation ofthe given arguments, how can the agents reach a collective compromise on howto evaluate those arguments?

Thus, the problem we face is that of judgment aggregation [21] in the con-text of argumentation frameworks. In particular, taking as an input a set ofindividual judgments as to how each argument in some argumentation frame-work AF = 〈A,⇀〉 must be labelled, we need to come up with a collectivejudgment. Given a set of agents Ag = {1, . . . , n}, if each agent i ∈ Ag has alabelling Li, we need to find an aggregation operator for AF , which we defineas a partial function3 F : L(AF )n → L(AF ), where L(AF ) is the class oflabellings of AF . This means that for each a ∈ A, [F (L)](a) is the collectivelabel assigned to a, if F is defined for L = (L1, . . . , Ln) 4.

4 Desirable Properties of Aggregation Operators

Aggregation involves comparing and assessing different points of view. Thereare, of course, many ways of doing this, as extensively discussed in the lit-erature of Social Choice Theory [16]. In this literature, a consensus on somenormative ideals has been reached, identifying what a ‘fair’ way of adding up

3 We state that the function is partial to allow for cases in which collective judgment maybe undefined (e.g. when there is a tie in voting).

4 Throughout the document, if there is no confusion, we will use L to denote the profile(L1, . . . , Ln). We also use L(a) to refer to (L1(a), . . . , Ln(a)), where a ∈ A.

Judgment Aggregation in Multi-Agent Argumentation 9

votes should be. So for instance, if everybody agrees, the outcome must reflectthat agreement; no single agent can impose her view on the aggregate; the ag-gregation should be performed in the same way in each possible case, etc. Theseinformal requirements can be formally stated as properties that F should sat-isfy [21][12]. In all of the following postulates, it is assumed that a fixed argu-mentation framework AF = 〈A,⇀〉 and a set of agents Ag = {1, . . . , n} aregiven. The postulates can be grouped as follows: 5

Group 1: Domain and co-domain postulates

In judgment aggregation, two postulates that are commonly assumed arethose of Universal Domain and Collective Rationality. The former requires thatany profile of labellings chosen from a pre-specified set of feasible labellingscan be used as input to F and F will return an answer. The question is: whatdo we take to be the set of feasible labellings in our setting? This depends onwhich semantics we assume is being used. Theoretically we can have a differ-ent version of Universal Domain for each semantics. However since completesemantics represent consistent and self-defending points of views, it representsthe best counterpart for the logical consistency in Judgment Aggregation:

Universal Domain F can take as input all profiles L = (L1, . . . , Ln)such that L ∈ Comp(AF )

n

Similarly we could have a different version of Collective Rationality - onefor each semantics - stating that the output of the aggregation should also befeasible. Again, since we focus on complete semantics, we focus on the followingversion:

Collective Rationality For all profiles L, F (L) ∈ Comp(AF ).

Later, in Section 7, we will break this postulate down into further constituents.Note that although the domain for L is not restricted here, we will mostly fo-cus on the case where profiles L ∈ Comp(AF )

n. However, in Subsection 8.2

we will use other semantics as a domain for L.

Group 2: Fundamental postulates

Next we come to the standard property that forms the cornerstone of theusual impossibility results in judgment aggregation. It says the collective labelof an argument depends only on the votes on that argument, independent ofthe other arguments.

Independence For any two profiles L = (L1, . . . , Ln), L′ = (L′1, . . . , L′n)

and argument a ∈ A, if Li(a) = L′i(a) for all i ∈ Ag then [F (L)](a) =[F (L′)](a).

5 This style of presentation of postulates was inspired by [18] which is on binary aggrega-tion.

10 Edmond Awad et al.

The effect of Independence is that aggregation is done “argument-by-argument”.To be slightly more precise, each argument a ∈ A essentially has its own ag-gregation operator Ia associated to it, that takes an n-tuple of labels x =(l1, . . . , ln) as input (representing the “vote” of each agent on the label of a)and returns another label Ia(x) as output (the “collective label”) of a. Then[F (L)](a) = Ia((L1(a), . . . , Ln(a))).

Next, we have Anonymity, which says the identity of which agent submitswhich labelling is irrelevant.

Anonymity For any profile L = (L1, . . . , Ln), if L′ = (Lσ(1), . . . , Lσ(n))for some permutation σ on Ag , then F (L) = F (L′).

If we add Anonymity to Independence, then it means the outputs of the func-tions Ia described above depend only on the number of votes that each labelgets in x. Essentially it means Ia outputs a collective label just taking as inputthe triple (#in,#out,#undec) of numbers denoting, respectively, the numberof votes for in, out and undec in x.

Proposition 1 Let F be an aggregation operator. Then F satisfies both In-dependence and Anonymity iff for each a ∈ A there exists a function Ia :N3 → {in, out, undec} such that, for all L we have [F (L)](a) = Ia(#in,#out,#undec).

Proof (Outline) . The “if” case is straightforward, since permuting the rowsdoes not change the vote distribution and so Anonymity will hold. Indepen-dence is also clear.

For the “only if” case, Independence gives us the existence of the func-tion Ia such that [F (L)](a) = Ia(L1(a), . . . , Ln(a)) and then Anonymity im-plies that two vectors that have the same vote distribution will give the sameresults, so we can set Ia(#in,#out,#undec) = Ia(L1(a), . . . , Ln(a)) where(L1(a), . . . , Ln(a)) is any vote which has (#in,#out,#undec) as its distribu-tion.

A weakening of Anonymity is Non-Dictatorship:

Non-Dictatorship There is no i ∈ Ag such that, for every profileL = (L1, . . . , Ln) we have F (L) = Li.

Group 3: Unanimity postulates

Next we move to Unanimity, and some other postulates related to it.

Unanimity If L is such that there exist some L s.t. Li = L for alli ∈ Ag then F (L) = L.

This postulate is also familiar from judgment aggregation, but the move to3-valued labellings rather than the 2 usually seen in judgment aggregationopens up the possibility to define other variants of Unanimity, one of which isused by Dokow and Holzman [14], called Supportiveness:

Judgment Aggregation in Multi-Agent Argumentation 11

Supportiveness For any profile L and all a ∈ A, there exists i ∈ Agsuch that [F (L)](a) = Li(a).

Supportiveness says that, for each argument a and label l, the collective judg-ment cannot be set to l without at least one agent voting for that l. ClearlySupportiveness implies Unanimity.

Group 4: Systematicity postulates

Now we come to the Systematicity postulates which deal with neutralityissues across arguments and labels. We can list two variants, both of whichimply Independence. We start with the stronger version:

Strong Systematicity For any two profiles L = (L1, . . . , Ln) andL′ = (L′1, . . . , L

′n) and arguments a, b ∈ A, and for every permutation

ρ on the set of labels {in, out, undec}, if ∀i ∈ Ag : Li(a) = ρ(L′i(b))then [F (L)](a) = ρ([F (L′)](b)).

To illustrate Strong Systematicity, consider the example in Figure 5. Wehave the following three labellings: L1 = ({a}, {b}, {}), L2 = ({b}, {a}, {}),L3 = ({}, {}, {a, b}).

a b

in out

L2

a b

L1

a b

L3

undec

Fig. 5 An example illustrating Strong Systematicity.

Consider the profiles L = (L1, L1, L2, L3) and L′ = (L3, L3, L1, L2). Then,L(a) = (in, in, out, undec) and L′(a) = (undec, undec, in, out). Let ρ be thepermutations on labels such that ρ(in) = undec, ρ(out) = in, and ρ(undec) =out. Then, we can see that in this example ∀i ∈ Ag : L′i(a) = ρ(Li(a)). StrongSystematicity requires that [F (L′)](a) = ρ([F (L)](a)).The postulate forces us to give an even-handed treatment to the labels in, outand undec. This makes sense if we consider in, out and undec as three inde-pendent labels. However, one might be tempted to consider undec as a middlelabel between in and out. Hence, the equal treatment might not be desirablein this case. One might suggest a version of Systematicity that treats in andout equally. We define this version (which we call in/out-Systematicity) in alater section. in/out-Systematicity lies in the middle between Strong System-aticity and the following version of Systematicity which can be obtained byrestricting the class of permutations, until we only consider the identity.

12 Edmond Awad et al.

Weak Systematicity For any two profiles L = (L1, . . . , Ln) and L′ =(L′1, . . . , L

′n) and arguments a, b ∈ A, if ∀i ∈ Ag : Li(a) = L′i(b) then

[F (L)](a) = [F (L′)](b).

Clearly Independence follows from Weak Systematicity by just setting a =b. If we strengthen Independence to Weak Systematicity then the functions Ia,mentioned earlier, are identical for all arguments.

Group 5: Monotonicity postulates

Our final group relates to Monotonicity.

Monotonicity Let la ∈ {in, out, undec} be such that given two pro-files L = (L1, . . . , Li, . . . , Li+k, . . . , Ln) and L′ = (L1, . . . , L

′i, . . . , L

′i+k, . . . , Ln)

(differing only in the labellings of agents i, i + 1, . . . , i + k) where i ∈{1, . . . , n} and k ∈ {0, . . . , n− i}, if Lj(a) 6= la while L

′

j(a) = la for allj ∈ {i, . . . , i+ k}, then [F (L)](a) = la implies that [F (L′)](a) = la.

Monotonicity states that if a set of agents switch their label of argumenta to the collective label of a then the collective label of a remains the same.

5 The Argument-Wise Plurality Rule

An obvious candidate aggregation operator to check out is the plurality votingoperator M . In this section, we analyse a number of key properties of thisoperator. Intuitively, for each argument, it selects the label that appears mostfrequently in the individual labellings.

Definition 6 (Argument-Wise Plurality Rule (AWPR)) Let AF =〈A,⇀〉 be an argumentation framework. Given any argument a ∈ A and anyprofile L = (L1, . . . , Ln), then [M(L)](a) = la ∈ {in, out, undec} iff

|{i : Li(a) = la}| > maxl′a 6=la

|{i : Li(a) = l′a}|

Example 5 (Three Detectives (cont.)) Continuing on Example 2, applying theargument-wise plurality rule, we have:

– [M((L1, L2, L3))](a1) = out

– [M((L1, L2, L3))](a2) = in

– [M((L1, L2, L3))](a3) = out

Note that M is defined for all profiles that cause no ties.

5.1 Properties of Argument-Wise Plurality Rule

We now analyse whether AWPR satisfies the properties listed above.

Judgment Aggregation in Multi-Agent Argumentation 13

Theorem 1 The argument-wise plurality rule operator M satisfies Support-iveness, Anonymity, Strong Systematicity, and Monotonicity.

Proof – Supportiveness: consider any profile L = (L1, . . . , Ln). Suppose, to-wards a contradiction, that for some argument a, there exists no agent isuch that Li(a) = la where la = [M(L)](a). Then |{i : Li(a) = la}| = 0.But, |{i : Li(a) = la}| > maxl′a 6=la |{i : Li(a) = l′a}| > 0. Contradiction.

– Anonymity: consider any profile L = (L1, . . . , Ln). [M(L)](a) = la if andonly if |{i : Li(a) = la}| > maxl′a 6=la |{i : Li(a) = l′a}| if and only if |{ρ(i) :Lρ(i)(a) = la}| > maxl′a 6=la |{ρ(i) : Lρ(i)(a) = l′a}|, which is equivalent to[M((Lρ(1), . . . , Lρ(i), . . . , Lρ(n)))](a) = la.

– Strong Systematicity: consider, for any two profiles L = (L1, . . . , Ln) andL′ = (L′1, . . . , L

′n), and for any a, b ∈ A, the permutation ρ : {in, out, undec}

→ {in, out, undec}. Suppose, towards a contradiction, that for any i, Li(a) =ρ(L′i(b)), and [M(L)](a) = la but ρ(M(L′)[b]) 6= ρ(la). But then,|{i :

Li(a) = la}| = |{i : L′

i(b) = ρ(la)}| while for any l′a 6= la, |{i : Li(a) =

l′a}| = |{i : L′

i(b) = ρ(l′a)}|. So, if |{i : Li(a) = la}| > maxl′a 6=la |{i : Li(a) =

l′a}| then, we have |{i : L′

i(b) = ρ(la)}| > maxl′a 6=la |{i : L′

i(b) = ρ(l′a)}| aswell. Contradiction.

– Monotonicity: Consider the following two profiles L = (L1, . . . , Li, . . . , Li+k,. . . , Ln) and L′ = (L1, . . . , L

′i, . . . , L

′i+k, . . . , Ln) (differing only in the la-

bellings of agents i, i+1, . . . , i+k) where i ∈ {1, . . . , n} and k ∈ {0, . . . , n−i}. Suppose, towards a contradiction, that for a ∈ A and a label la wehave that Lh(a) 6= la while L

′

h(a) = la for all h ∈ {i, . . . , i + k}, and wehave that [M(L)](a) = la while [M(L)](a) 6= la. But then, |{j : Lj(a) =la}| > maxl′a 6=la |{j : Lj(a) = l′a}| in the profile L while in the profile

(L1, . . . , Ln)≡L′, we have {j : Lj(a) = la} = {j : Lj(a) = la}∪{i, . . . , i+k}and {j : Lj(a) = l′a} ⊆ {j : Lj(a) = l′a} for every other labelling l′a. Then

|{j : Lj(a) = la}| > maxl′a 6=la |{j : Lj(a) = l′a}|. Contradiction.

Corollary 1 The argument-wise plurality rule operator M satisfies Unanim-ity, Weak Systematicity, Independence, and Non-Dictatorship.

Proof – Unanimity. Follows from Supportiveness.– Weak Systematicity. Follows from Strong Systematicity.– Independence. Follows from Strong Systematicity.– Non-Dictatorship. Follows from Anonymity.

Note that our definition for Supportiveness is taken over arguments (sim-ilar to its counterpart in Judgment Aggregation literature). One should notconfuse this definition with Supportiveness taken over labellings i.e. the collec-tive labelling for an AF is supported by at least one agent’s labelling. WhileAWPR satisfies Supportiveness over arguments, it violates Supportiveness overlabellings. It is possible to construct cases in which the aggregated labellingdoes not coincide with any one of the individual labellings.

To see how, suppose that there are seven arguments a, b1, b2, b3, c1, c2, c3,with the attack relation depicted in Figure 6. Suppose we have three agentswhose labellings are L1, L2, and L3 (as shown in the figure). Then:

14 Edmond Awad et al.

– [M((L1, L2, L3))](b1) = in, [M((L1, L2, L3))](c1) = out,– [M((L1, L2, L3))](b2) = in, [M((L1, L2, L3))](c2) = out,– [M((L1, L2, L3))](b3) = in, [M((L1, L2, L3))](c3) = out, and– [M((L1, L2, L3))](a) = out.

That is, M((L1, L2, L3)) is different from L1, L2 and L3.

in out

ab2

b1c1

b3

c2

c3

ab2c2

b3

c1

c3

b1

a

b3c3

b2

c1

c2

b1

ac2 b2

c1

c3

b1

b3

L1 L2 L3

M(L1, L2, L3)

Fig. 6 An example which shows that an aggregated labelling may not coincide with any ofthe individual labellings.

Despite all these promising results, it turns out that plurality operatorviolates Universal Domain and Collective Rationality postulates. Since AWPRis not defined for profiles that cause ties, then M cannot take as input anyprofile L ∈ Comp(AF )

n. Then, it violates Universal Domain. A weaker version

of Universal Domain can be defined.

No-Tie Universal Domain An aggregation operator F can take asinput all profiles L = (L1, . . . , Ln) such that L does not cause a tie andL ∈ Comp(AF )

n.

Since there are no restrictions (other than having no ties) on how labellingsare defined, AWPR satisfies No-Tie Universal Domain.

As for Collective Rationality, consider the following example.

Example 6 Suppose argument c has two defeaters, a and b, and argument a(resp. b) defeats and is defeated by argument a′ (resp. b′). Suppose we have 3agents, with votes as shown in Figure 7. We have [M(L)](c) = out, but it isnot the case that [M(L)](a) = in or [M(L)](b) = in.

Judgment Aggregation in Multi-Agent Argumentation 15

c

b

a

in out

c

Agent 2Agent 1 Agent 3

c

a

Outcome

a'

b'

a'

a

a'

b

b'

c

a

a'

b

b'

b

b'

Fig. 7 Counterexample to Collective Rationality

Interestingly, the above counterexample demonstrates a variant of the dis-cursive dilemma [21] in the context of argument evaluation, which itself is avariant of the well-known Condorcet paradox.

5.2 Introducing Tie-Breaking Rules to Satisfy Universal Domain

As shown earlier, among the ten defined postulates, the argument-wise plural-ity rule only violates Universal Domain and Collective Rationality which arevery fundamental postulates. One might be tempted to try to make AWPRsatisfies Universal Domain by adding a deterministic tie-breaking rule to dealwith ties. We define a new version of AWPR (a version that deals with ties):

Definition 7 (AWPR with a Tie-Breaking Rule (AWPR w/TBR ))Let AF = 〈A,⇀〉 be an argumentation framework. Let T : {in, out, undec}n →{in, out, undec} be a deterministic tie-breaking rule. Then aggregation oper-ator MT is defined by setting, for each profile L and argument a ∈ A,

[MT (L)](a) =

{[M(L)](a) if this existsT ((L1(a), . . . , Ln(a))) otherwise

One can show that AWPR w/TBR satisfies Universal Domain since thereare no restrictions on how labellings are defined.

Which tie-breaking rule could we use? There are several possibilities. Thefirst one would be to “default” to some fixed label, with an obvious choicebeing undec. Intuitively, when agents’ votes have a tie, everyone would behappy with undec. However, the use of this rule would result in violatingSupportiveness, Strong Systematicity, and Monotonicity as we confirm in thefollowing example. Let Mu be the AWPR (i.e. M) but with the use of “chooseundec with ties” rule. Consider the example in Figure 8. We have the followingthree labellings: L1 = ({a}, {b}, {}), L2 = ({b}, {a}, {}), L3 = ({}, {}, {a, b}).

– Supportiveness. [Mu((L1, L2))](a) = undec while L1(a) 6= undec and L2(a) 6=undec.

16 Edmond Awad et al.

a b

in out

L2

a b

L1

a b

L3

undec

Fig. 8 An example shows how using AWPR with “choose undec with ties” rule violatesSupportiveness, Strong Systematicity, and Monotonicity.

– Strong Systematicity. Let ρ be the permutations on labels such that ρ(in) =in, ρ(out) = undec, and ρ(undec) = out (i.e. just swap out and undec),and Consider [Mu((L1, L3))](a). Note that we have L1(a) = ρ(L1(a))and L3(a) = ρ(L2(a)). Strong Systematicity requires [Mu((L1, L3))](a) =ρ([Mu((L1, L2))](a)) = ρ(undec) = out. However, [Mu((L1, L3))](a) =undec.

– Monotonicity. We have [Mu((L1, L1, L2, L2))](a) = undec. If first agentswitches from L1 to L3 (where L3(a) = undec), then Monotonicity requiresthat [Mu((L3, L1, L2, L2))](a) = undec. However, [Mu((L3, L1, L2, L2))](a) =out.

The violation of Supportiveness, Strong Systematicity, and Monotonicity ismainly caused by the special treatment of undec. However, treating undec ina special way might not be a problem in some cases. In these cases, one shouldat least maintain the neutrality between in and out. Hence, one can redefinethe previous three properties for the convenience of Mu. Following, we definein/out-Supportiveness, in/out-Systematicity, and in/out-Monotonicity.

in/out-Supportiveness For any profile L = (L1, . . . , Ln) and a ∈ A, if[F (L)](a) 6= undec then there exists some agent i such that [F (L)](a) =Li(a).

in/out-Systematicity For any 2 profiles L = (L1, . . . , Ln) and L′ =(L′1, . . . , L

′n) and arguments a, b ∈ A, and for every undec-preserving

permutation ρ on the set of labels {in, out, undec} (i.e. ρ(undec) =undec), if ∀i ∈ Ag : Li(a) = ρ(L′i(b)) then [F (L)](a) = ρ([F (L′)](b)).

in/out-Monotonicity Let la ∈ {in, out} be such that given two pro-files L = (L1, . . . , Li, . . . , Li+k, . . . , Ln) and L′ = (L1, . . . , L

′i, . . . , L

′i+k,

. . . , Ln) (differing only in the labellings of agents i, i+1, . . . , i+k) wherei ∈ {1, . . . , n} and k ∈ {0, . . . , n− i}, if Lj(a) 6= la while L

′

j(a) = la forall j ∈ {i, . . . , i+ k}, then [F (L)](a) = la implies that [F (L′)](a) = la.

Judgment Aggregation in Multi-Agent Argumentation 17

The following result confirms that Mu does indeed satisfy these modified pos-tulates.

Proposition 2 Mu satisfies in/out-Supportiveness, in/out-Systematicity andin/out-Monotonicity.

Proof (Outline)

– in/out-Supportiveness: If [Mu(L)](a) 6= undec then it must be the casethat [Mu(L)](a) = [M(L)](a) and so there must be at least one i such that[Mu(L)](a) = Li(a).

– in/out-Systematicity: Let ρ be an undec-preserving permutation on {in, out, undec}such that Li(a) = ρ(L′i(b)) for all i ∈ Ag. There are two possibilities for ρ.Either (i) it equals the identity, or (ii) it just swaps in and out.In case (i) we know the set - and hence the number - of agents who votein (respectively out, undec) for a in L is the same as the set - and hencethe number - of agents who vote in (respectively out, undec) for b in L′.Given this it is clear that [Mu(L)](a) = [Mu(L′)](b), i.e., (since ρ is theidentity) [Mu(L)](a) = ρ([Mu(L′)](b)).In case (ii) we know the set - and hence the number - of agents who vote in

(respectively out, undec) for a in L is the same as the set - and hence thenumber - of agents who vote out (respectively in, undec) for b in L′. Thismeans that in, respectively out, undec, is a plurality winner for a in L iffout, respectively in, undec, is a plurality winner for b in L′. So either thereis a plurality winner for both a in L and for b in L′, in which case we obtain[Mu(L)](a) = ρ([Mu(L′)](b)), or there is neither a plurality winner for ain L, nor for b in L′, in which case [Mu(L)](a) = undec = [Mu(L′)](b) =ρ([Mu(L′)](b)) (this last step using the fact that ρ is undec-preserving).

– in/out-Monotonicity: Let L, L′ be as in the statement of the postulateand suppose [Mu(L)](a) ∈ {in, out}. Then [Mu(L)](a) = [M(L)](a) bydefinition of Mu. Using the fact that M satisfies Monotonicity we know[M(L′)](a) = [M(L)](a). Since a plurality winner for a exists in L′, wemust have [Mu(L′)](a) = [M(L′)](a) and so [Mu(L′)](a) = [Mu(L)](a) asrequired.

Two more possible tie-breaking rules would be to let one agent decidein case of a tie, or to introduce a special symbol to represent a tie label. Anexample of the first case might exist in some committees where the chairpersonresolves ties. Unfortunately, the use of this rule obviously violates Anonymity.

Let Md be the AWPR (i.e. M) but with the use of “let the same agentdecides” rule.

– Anonymity. Consider the example in Figure 8. Let the first agent always de-cides in case of ties. Then, we have [Md(L1, L2)](a) = in. Now Anonymityrequires that [Md(L2, L1)](a) = in. However, [Md(L2, L1)](a) = out.

The second case is to use a special tie-symbol (such as ∗) to represent atie label. This was suggested by Dokow and Holzman [13], but for the case

18 Edmond Awad et al.

of binary labelling, where ∗ resembles abstention. However, one can extendit to the ternary case: Use ∗ as a labelling for an argument whenever a tiehappens. Then, check for each argument with ∗, whether it has only one validlabelling (given its defeaters labellings). Keep repeating the process until everyargument with ∗ can have more than one valid labelling. However, this processis not guaranteed to terminate and it obviously violates Independence. Anotherprocedure can be to choose one fixed label to fill every ∗, but this implies aspecial treatment for one of the labels in, out, or undec, and hence a violationof Strong Systematicity.

As we see, all the three rules violate either Strong Systematicity or Anonymity.However, the three previous rules are not exclusive, there can be different oth-ers. Then, an interesting question is “Does there exist an operator that satisfiesUniversal Domain, Strong Systematicity, and Anonymity?”. In the next sec-tion, we show that the answer for this question is negative. 6

6 The Impossibility of Good Aggregation Operators

In the previous section, we analysed a particular judgment aggregation oper-ator (namely, argument-wise plurality rule). We showed that while it satisfiesmost key properties, it fails to satisfy Universal Domain and Collective Ra-tionality. Then, we tried to make the AWPR satisfy Universal Domain byadding a deterministic tie-breaking rule. This produced a new operator, whichwe named AWPR w/TBR. However, we failed to suggest a good tie-breakingrule that satisfies both Strong Systematicity and Anonymity. In this sectionwe show that, depending on the argumentation framework and the number ofagents, it is impossible to have such a rule.

Theorem 2 There exists an argumentation framework AF such that, for anyset of agents that is divisible by three, there exists no labelling aggregationoperator satisfying Universal Domain, Anonymity and Strong Systematicity.

Proof It is enough to assume an AF that contains at least one argument awhich can feasibly take on any label, i.e. there exist complete labellings Lin, Lout

and Lundec over AF such that Lin(a) = in, Lout(a) = out and Lundec(a) =undec. Divide n agents into 3 groups G1, G2, G3 of equal size. By UniversalDomain, all profiles consisting of legal labellings are valid input. Assume aprofile in which everyone in G1 provides labelling Lin, everyone in G2 providesLout and everyone in G3 provides Lundec. For now let’s denote this profile byL = ([G1 : Lin], [G2 : Lout], [G3 : Lundec]). Now, assume for contradiction thatF is an aggregation operator for AF satisfying Universal Domain, Anonymityand Strong Systematicity. Let ρ : {in, out, undec} → {in, out, undec} be anypermutation on the set of labels such that ρ(l) 6= l for all labels l (for instance,ρ(in) = out, ρ(out) = undec, ρ(undec) = in), and let L′ denote the profile

6 Note that we do not consider the use of a non-deterministic tie-breaking rule since ithas its own issues too, such as producing different outcomes given the same profile.

Judgment Aggregation in Multi-Agent Argumentation 19

([G1 : Lρ(in)], [G2 : Lρ(out)], [G3 : Lρ(undec)]). Since L′i(a) = ρ(Li(a)) for alli ∈ Ag, Strong Systematicity implies [F (L′)](a) = ρ([F (L)](a)). However, wechose ρ s.t. ρ(l) 6= l. Hence, [F (L′)](a) 6= [F (L)](a). But Anonymity implies[F (L)](a) = [F (L′)](a). Contradiction. Hence no such F can exist.

The previous result can be read in two ways: First, the AWPR cannotbe made to satisfy Universal Domain without violating Strong Systematic-ity or Anonymity. Second, there exists no aggregation operator that satisfiesUniversal Domain, Strong Systematicity and Anonymity.

Note that the previous theorem was stated for a set of agents divisible bythree. Hence, one might wonder whether we could rule out the possibility of3-way ties, by assuming n cannot be a multiple of three. This is possible, butthen with even number of agents, we can show that there is still a large class ofAF s which do not have an operator satisfying those three postulates withoutviolating Collective Rationality.

Theorem 3 There exists an argumentation framework AF such that, for anyset of agents of even cardinality, there exists no labelling aggregation operatorsatisfying Universal Domain, Anonymity, Strong Systematicity and CollectiveRationality.

Proof It is enough to assume an AF that contains at least one argument athat can feasibly take on just two out of the three possible labels. For concrete-ness suppose a can only take on labels out and undec. (An example of sucha framework and an argument can be seen in the proof of Theorem 4 below,in which c can only be either out or undec). Let Lundec and Lout be two com-plete labellings such that Lundec(a) = undec and Lout(a) = out. Divide theagents into two groups G1, G2 of equal size. By Universal Domain, all pro-files consisting of legal labellings are valid input, so assume a profile in whicheveryone in G1 provides labelling Lundec and everyone in G2 provides Lout.Denote the resulting profile by L = ([G1 : Lundec], [G2 : Lout]) and assume forcontradiction that F is an aggregation operator for this AF that satisfies Uni-versal Domain, Anonymity, Strong Systematicity and Collective Rationality.Let ρ be the permutation that swaps undec and out, i.e., ρ(undec) = out andρ(out) = undec, and let L′ = ([G1 : Lout], [G2 : Lundec]). By Anonymity weknow [F (L)](a) = [F (L′)](a). Then it cannot be that [F (L)](a) = undec, forif so then Strong Systematicity would imply [F (L′)](a) = ρ(undec) = out 6=[F (L)](a), and similarly it cannot be that [F (L)](a) = out. Thus we musthave [F (L)](a) = in. But by Collective Rationality [F (L)](a) ∈ {undec, out}.Contradiction.

The careful reader can realize that Collective Rationality can be substi-tuted with Supportiveness in the previous theorem. As for the proof, the lastsentence becomes: “Thus we must have [F (L)](a) = in. But by Supportiveness[F (L)](a) ∈ {undec, out}. Contradiction”.

However, one might argue that Strong Systematicity is quite a strong con-dition. Treating in, out, and undec differently can be tolerated. Then, it is

20 Edmond Awad et al.

interesting to ask: “Does there exist an operator that satisfies Universal Do-main, Weak Systematicity, and Anonymity?”. We certainly have no answer forthis question in the meantime. However, we can show that such an operator,if it exists, is not a perfect one. Following, we show that an operator thatsatisfies Universal Domain, Weak Systematicity, and Anonymity, if it exists,would violate Collective Rationality and/or Unanimity.

Theorem 4 There exists an argumentation framework AF such that, for anyset of agents of even cardinality, there exists no labelling aggregation opera-tor satisfying Universal Domain, Weak Systematicity, Anonymity, CollectiveRationality, and Unanimity.

Proof Consider the following argumentation framework. An argument c is de-feated by two arguments a and b which defeat each others.

c

b

a

in out undec

c

b

a

c

b

a

L L’ L’’

Consider the two labellings L = ({a}, {b, c}, {}) and L′ = ({b}, {a, c}, {}).Assume, towards a contradiction, there exists an aggregation operator F thatsatisfies Universal Domain, Collective Rationality, Weak Systematicity, Anonymityand Unanimity.

By Universal Domain, we may consider any profile consisting of legal la-bellings. Consider the two profiles L = (L, . . . , L, L′, . . . , L′) and L′ = (L′, . . . , L′, L, . . . , L).That is, in L half the agents give L and the other half give L′, and then in L′the agents switch from L to L′ and vice versa.

By Unanimity we know

[F (L)](c) = out. (2a)

By Weak Systematicity we also know [F (L)](a) = [F (L′)](b). But since L andL′ are permutations of each other we know F (L) = F (L′) by Anonymity andso we obtain

[F (L)](a) = [F (L)](b). (2b)

But there is no complete labelling simultaneously satisfying (2a) and (2b).Contradiction. Hence no F can exist.

One might note that all of the above theorems exploit the use of profilesthat include ties. What if we relax Universal Domain to No-Tie Universal Do-main. Following, we show that an aggregation operator which satisfies No-Tie

Judgment Aggregation in Multi-Agent Argumentation 21

Universal Domain (but not necessarily Universal Domain) cannot also satisfyWeak Systematicity, Anonymity, Collective Rationality, and Supportiveness.

Theorem 5 There exists an argumentation framework AF such that, for anyset of agents that is divisible by three, there exists no labelling aggregation op-erator satisfying No-Tie Universal Domain, Weak Systematicity, Anonymity,Collective Rationality, and Supportiveness.

Proof Consider the following argumentation framework. An argument a is de-feated by two arguments b and c. Argument b (resp. c) defeats and is defeatedby argument b′ (resp. c′).

in out

a

c

bb’

c’

a

cc’

b’ b

c’

b

c

L3L2L1

a

b’

Consider the three labellings L1 = ({b, c′}, {a, b′, c}, {}), L2 = ({b′, c}, {a, b, c′},{}) and L3 = ({a, b′, c′}, {b, c}, {}).

Assume, towards a contradiction, there exists an aggregation operator Fthat satisfies No-Tie Universal Domain, Collective Rationality, Weak System-aticity, Anonymity and Supportiveness.

By No-Tie Universal Domain, we may consider any profile consisting oflegal labellings as long as it does not cause a tie. We consider here three agents,but the same proof can be shown for any set of agents that is divisible by three.Consider the three profiles L = (L1, L2, L3), L′ = (L′1, L

′2, L′3) = (L3, L1, L2)

and L′′ = (L′′1 , L′′2 , L

′′3) = (L2, L3, L1).

Since ∀i, Li(a) = L′i(c), then by Weak Systematicity we know:

[F (L)](a) = [F (L′)](c) (3a)

But since L and L′ are permutations of each other we know [F (L)] =[F (L′)] by Anonymity and so we obtain

[F (L)](c) = [F (L′)](c). (3b)

From Eq.3a and Eq.3b:

[F (L)](a) = [F (L)](c). (3c)

Similarly, since ∀i, Li(b) = L′′i (c), then by Weak Systematicity we know:

22 Edmond Awad et al.

[F (L)](b) = [F (L′′)](c) (3d)

But since L and L′′ are permutations of each other we know [F (L)] =[F (L′′)] by Anonymity and so we obtain

[F (L)](c) = [F (L′′)](c). (3e)

From Eq.3d and Eq.3e:

[F (L)](b) = [F (L)](c). (3f)

From Eq.3c and Eq.3f:

[F (L)](a) = [F (L)](b) = [F (L)](c). (3g)

The last equation suggests that a, b, and c have the same collective labelling.However, by Collective Rationality, the only legal labelling that satisfy Eq.3gis undec:

[F (L)](a) = [F (L)](b) = [F (L)](c) = undec. (3h)

However, F satisfies Supportiveness by assumption. Contradiction.

One can draw a connection between this result and the previous one. Re-laxing Universal Domain to No-Tie Universal Domain, introduces anotherimpossibility result, in which Unanimity is replaced with the stronger postu-late Supportiveness.

The above impossibility results highlight a major barrier to reaching goodcollective judgment about argument evaluation in general. These build onthe solid foundation of social choice and judgement aggregation for impossi-bility results such as Arrow’s celebrated impossibility theorem on preferenceaggregation [1] and List and Pettit’s impossibility theorem on judgment ag-gregation in propositional logic [21]. In our context, the result means that anaggregation on argument evaluation can only be achieved at a cost to somefew basic desirable properties. Unfortunately, there is no escape from violatingthese conditions or accepting irrational aggregate argument labellings withoutsomewhat lowering our standards in terms of desirable criteria.

7 Collective Rationality Postulates

In this section, we characterise Collective Rationality in terms of conditionsthat need to be satisfied by profiles. To do this, we need to go back to theconditions of legal (i.e. complete) labelling, since we have to ensure that theseconditions are satisfied in the outcome of an aggregation operator. For ease ofreference, we reformulate the postulates for legal labelling here (from Definition4) following the formulation in [9, Proposition 1]:

– ∀a ∈ A : if L(a) = in then ∀b ∈ A : ( if b ⇀ a then L(b) = out); and– ∀a ∈ A : if L(a) = out then ∃b ∈ A such that (b ⇀ a and L(b) = in);

Judgment Aggregation in Multi-Agent Argumentation 23

– ∀a ∈ A : if L(a) = undec then– ∃b ∈ A : (b ⇀ a and L(b) = undec); and– 6 ∃b ∈ A : (b ⇀ a and L(b) = in)

What we need to do now is identify conditions on the domain of the aggregationoperator that guarantee these postulates. So, to guarantee that the postulateholds, we need some restrictions.

We define the following condition, which we call IN-Collective Rationality(IN-CR). Intuitively, it requires that if an argument a is collectively acceptedby the agents, then the agents must collectively reject all counter-argumentsagainst a.

IN-Collective Rationality (IN-CR) For any profile L and a ∈ A,if [F (L)](a) = in then ∀b ∈ A, we have (b ⇀ a⇒ [F (L)](b) = out).

IN-CR, however, can be further characterised by two parts: IN-CR1 andIN-CR2.

IN-CR1 For any profile L and a ∈ A, if [F (L)](a) = in then @b ∈ A,such that b ⇀ a and [F (L)](b) = in.

IN-CR2 For any profile L and a ∈ A, if [F (L)](a) = in then @b ∈ A,such that b ⇀ a and [F (L)](b) = undec.

An aggregation operator satisfies IN-CR if and only if it satisfies both IN-CR1 and IN-CR2. Note that IN-CR1 represents the “collective” counterpartof the condition of conflict-freeness which is usually agreed on as a minimalreasonable condition in argument evaluation.

We present now the OUT-Collective Rationality (OUT-CR) condition. In-tuitively, this condition means that if an argument a is collectively rejected bythe agents, then the agents must also collectively agree on accepting at leastone of the counter-arguments against a. In other words, the agents’ individ-ual attacks on a are not arbitrary, but must exhibit some minimal form ofcoordination.

OUT-Collective Rationality (OUT-CR) For any profile L and a ∈A, if [F (L)](a) = out then ∃b ∈ A, such that b ⇀ a and [F (L)](b) = in.

The final condition we require has to do with the label undec. We presentnow the UNDEC-Collective Rationality (UNDEC-CR) condition. An argu-ment must be labelled undec if and only if: (i) it is not the case that all of itsdefeaters are out, that is, at least one of its defeaters is undec; and (ii) noneof its defeaters is in.

UNDEC-Collective Rationality (UNDEC-CR) For any profile Land a ∈ A, if [F (L)](a) = undec then:1. @b ∈ A, such that b ⇀ a and [F (L)](b) = in.2. ∃b ∈ A, such that b ⇀ a and [F (L)](b) = undec.

Similarly, UNDEC-CR can be further characterised by two parts: UNDEC-CR1 and UNDEC-CR2.

24 Edmond Awad et al.

UNDEC-CR1 For any profile L and a ∈ A, if [F (L)](a) = undec then@b ∈ A, such that b ⇀ a and [F (L)](b) = in.

UNDEC-CR2 For any profile L and a ∈ A, if [F (L)](a) = undec then∃b ∈ A, such that b ⇀ a and [F (L)](b) = undec.

An aggregation operator satisfies UNDEC-CR if and only if it satisfies bothUNDEC-CR1 and UNDEC-CR2.

Proposition 3 An argument aggregation operator F satisfies Collective Ra-tionality if and only if for each profile L = (L1, . . . , Ln) in its domain, itsatisfies the IN-CR, OUT-CR, and UNDEC-CR conditions.

8 Plurality Rule with Classical Semantics

In this section, we analyse the performance of AWPR with respect to CollectiveRationality when agents labellings are restricted to some classical semantics(i.e. complete, grounded, stable, semi-stable, and preferred). This investigationgives an entirely novel meaning to classical semantics in social choice settings.Rather than simply being compared by their logical rigour from the perspectiveof a single agent, semantics are compared based on the extent to which theyfacilitate collectively rational agreement among agents.

Our strategy will be based on the following approach. Since, by Proposi-tion 3, Collective Rationality arises iff IN-CR, OUT-CR, and UNDEC-CR aresatisfied, it is enough to check whether AWPR satisfies those properties.

8.1 Complete Semantics

Since the complete semantics generalizes other classical semantics, we provideanalysis for it first. Every property that is satisfied by AWPR with the com-plete semantics would be also satisfied by AWPR for the other other classicalsemantics.

Note that we have already shown earlier that AWPR violates CollectiveRationality (for some AF s) when agents’ labellings are restricted to completesemantics. In this section, we dig further to show which of the Collective Ra-tionality parts are violated.

It is very interesting to see that, as the lemma below shows, when agentscollectively accept an argument, the structure of the AWPR will ensure thatthey will not collectively accept any of its defeaters:

Lemma 1 AWPR satisfies IN-CR1. Using the argument-wise plurality rule,given any profile L = (L1, . . . , Ln), if an argument a is collectively accepted,none of its defeaters will be collectively accepted. Formally, if [M(L)](a) = in

for some arbitrary a ∈ A then @b ∈ A, such that b ⇀ a and [M(L)](b) = in.

Judgment Aggregation in Multi-Agent Argumentation 25

Proof Suppose that [M(L)](a) = in holds. By definition:

|{i : Li(a) = in}| > |{i : Li(a) = out}| (4a)

Since each Li is a legal labelling, an agent who votes in for a must also voteout for each defeater of a. Therefore:

∀b ⇀ a |{i : Li(b) = out}| ≥ |{i : Li(a) = in}| (4b)

We want to show that: @b ∈ A such that b ⇀ a and [M(L)](b) = in

Assume (towards contradiction) that the contrary holds. That is, ∃b′ ∈ Asuch that b

′⇀ a and [M(L)](b′) = in. Then:

|{i : Li(b′) = in}| > |{i : Li(b

′) = out}| (4c)

Since every agent who voted in for b′

would have voted out for a, we have:

|{i : Li(a) = out}| ≥ |{i : Li(b′) = in}| (4d)

By Eq.4c and Eq.4d:

|{i : Li(a) = out}| > |{i : Li(b′) = out}| (4e)

while from Eq.4b and Eq.4e we have that:

|{i : Li(a) = out}| > |{i : Li(a) = in}| (4f)

But this contradicts Eq.4a and the assumption that [M(L)](a) = in.

It is important to recognise that Lemma 1 is a non-trivial result. It showsthat, with AWPR, the postulate IN-CR1 is satisfied. This means, as we men-tioned earlier, that AWPR satisfies the “collective” version of conflict-freeness,a condition that is usually agreed on as a minimal reasonable condition in ar-gument evaluation. This comes “for free” as a result of the intrinsic structureof the individual labellings, leading to coordinated votes. Note, however, thatthe postulate is not fully satisfied. Although Lemma 1 guarantees that a col-lectively accepted argument will never have a collectively accepted defeater,it does not guarantee IN-CR2, that none of its defeaters will be collectivelyundecided. This is demonstrated in the following remark.

Remark 1 AWPR violates IN-CR2. If an argument is collectively accepted,some of its defeaters might be collectively undecided.

Proof Suppose argument c has two defeaters, a and b. Suppose we have 7agents, with votes as shown in Figure 9. Clearly, while c is collectively acceptedbecause [M(L)](c) = in, one of its defeaters is not collectively rejected because[M(L)](b) = undec.

As we saw earlier in Example 6, OUT-CR is violated by AWPR.

Remark 2 AWPR violates OUT-CR. If an argument is collectively rejected, itis not guaranteed that one of its defeaters will be collectively accepted.

26 Edmond Awad et al.

c

b

a

in out undec

c

b

a

c

b

a

Agent 4-5Agent 1-3 Agent 6-7

c

b

a

Outcome

a'a'

b' b' b' b'

a' a'

Fig. 9 Seven votes collectively accepting c, without collectively rejecting b

Proof See Example 6 for a counterexample.

The following remark shows that there are no intrinsic guarantees for sat-isfying UNDEC-CR1.

Remark 3 AWPR violates UNDEC-CR1. If an argument is collectively unde-cided, it is possible that one of its defeaters will be collectively accepted.

Proof Suppose argument c has two defeaters, a and b. Suppose we have 7agents. Suppose the votes are as shown in Figure 10. We have [M(L)](c) =undec with 4 votes, but we have [M(L)](a) = in with 3 votes, thus violatingthe postulate.

a

in out undec

Agent 3-4Agent 1-2 Agent 5-7

a

Outcome

a' a'

b

b'

b

b'

c cc

a

a'

b

b'

c

b

b'

a

a'

Fig. 10 Seven agents collectively undecided on c, but collective accepting a

Judgment Aggregation in Multi-Agent Argumentation 27

Similarly, the remark below shows that UNDEC-CR2 is not intrinsicallyguaranteed.

Remark 4 AWPR violates UNDEC-CR2. If an argument is collectively unde-cided, it is possible that none of its defeaters will be collectively undecided.

Proof Suppose argument c has two defeaters, a and b. Suppose we have 3agents, with votes as shown in Figure 11. Clearly, we have [M(L)](c) = undec,but we have [M(L)](a) = out and [M(L)](b) = out, which would have requiredc to be in.

c

b

a

in out undec

b

a

c

a

Agent 2Agent 1 Agent 3

a

Outcome

a'a'

b' b'

a' a'

b

b'

b

b'

c c

Fig. 11 Three votes collectively undecided about c, but not collectively undecided aboutany of its defeaters a or b.

So far, we undertook a detailed investigation of which of the properties(defined in the previous section) that characterise Collective Rationality areintrinsically satisfied due to structural properties of the argument completelabelling domain under plurality rule. Our finding is quite disappointing. Withthe exception of the IN-CR1 shown in Lemma 1, the remarks above show ableak picture.

8.2 Other Classical Semantics

Throughout this work, and especially in the previous part, we have assumed(sometimes implicitly) that each agent is required to choose one possible com-plete labelling. This is a valid requirement since each possible complete la-belling represents a valid self-defending viewpoint, that satisfies the widelyaccepted admissibility criterion. However, some may argue that other classicalsemantics (considered as a domain to choose from) might be more desirablein some cases. For example, sometimes, it might be more desirable for indi-viduals to have more committed viewpoints (i.e. to maximize in-labelled or

28 Edmond Awad et al.

out-labelled arguments), or to be restricted to labellings that minimize uncer-tainty (i.e. to minimize undec-labelled arguments).

Since the classical semantics, we consider here, add more restrictions tothe complete labelling, using any of these semantics can be considered as adomain restriction. In other words, choosing other classical semantics impliesa relaxation for No-Tie Universal Domain. Then, one may ask, “if agents wererestricted to other classical semantics, would the argument-wise plurality rulefind an escape route from the impossibility results?”. If that were the case,different semantics would find a novel field of application in the aggregationof labellings.

In this subsection, we provide an analysis for the grounded, stable, semi-stable, and preferred semantics as more restricted forms of labellings to choosefrom. We mainly show whether the Collective Rationality would be satisfiedunder any of these semantics. Note that the stable semantics would restrictthe topology of the graph, since stable semantics do not always exist.

The following proposition looks trivial but, as we will see, it is the mostpositive result in this subsection.

Proposition 4 If for every argument, agents can only vote for the groundedlabelling, then M satisfies IN-CR1, IN-CR2, OUT-CR, UNDEC-CR1 andUNDEC-CR2. Equivalently, M satisfies Collective Rationality.

Proof There always exists exactly one grounded extension [15]. Since completeextensions and complete labellings have a one-to-one relationship [8, Theorem1], then there always exists one grounded labelling (the grounded labelling isalso a complete labelling). Hence, if agents are restricted to grounded seman-tics, all agents will vote for the (only) grounded labelling for each argument. ByUnanimity, the collective labelling of every argument is the grounded labellingof this argument, which is a valid (complete) labelling. The Collective Rational-ity is trivially satisfied. By Proposition 3, IN-CR, OUT-CR, and UNDEC-CRare all satisfied. Equivalently, IN-CR1, IN-CR2, OUT-CR, UNDEC-CR1 andUNDEC-CR2 are all satisfied.

As a corollary for Lemma 1, when agents votes are restricted to stable(respectively semi-stable or preferred) labellings, AWPR satisfies IN-CR1.

Corollary 2 When agents can only vote for stable (respectively semi-stableor preferred) labellings, AWPR satisfies IN-CR1

Proof From Lemma 1, if agents can only vote for complete labellings, thenAWPR satisfies IN-CR1. Since every stable (respectively semi-stable or pre-ferred) labelling is a complete labelling, then when agents votes are restrictedto these semantics, AWPR satisfies IN-CR1.

Lemma 2 When agents can only vote for a stable labelling, AWPR satis-fies IN-CR2. If an argument is collectively accepted, none of its defeaters iscollectively undecided.

Judgment Aggregation in Multi-Agent Argumentation 29

Proof Suppose, towards a contradiction, that there exists an argument that iscollectively accepted and one of its defeaters is collectively undecided. Then,by Supportiveness, there exists one submitted labelling (by some agent) inwhich this argument is undecided. However, agents are only allowed to submita stable labelling, and stable labellings have no argument labelled undecided.Contradiction.

Remark 5 When agents can only vote for stable (respectively semi-stable orpreferred) labellings, AWPR violates OUT-CR. If an argument is collectivelyrejected, it is possible that none of its defeaters is collectively accepted.

Proof See Example 6 for a counterexample.

Lemma 3 When agents can only vote for a stable labelling, AWPR satisfiesUNDEC-CR (i.e. it satisfies both UNDEC-CR1 and UNDEC-CR2). If anargument is collectively undecided, none of its defeaters is collectively accepted,and at least one of its defeaters is collectively undecided.

Proof Since in stable labelling no argument is labelled undecided, by Support-iveness, there is no argument that is collectively undecided. Then, this lemmaholds.

We continue with the semi-stable and preferred semantics.

Remark 6 When agents can only vote for a semi-stable (respectively preferred)labelling, AWPR violates IN-CR2. If an argument is collectively accepted, itis possible that one of its defeaters is collectively undecided.

Proof Suppose argument c4 has two defeaters, a4 and c6. Suppose we have 7agents, with votes as shown in Figure 12. Clearly, while c4 is collectively ac-cepted because [M(L)](c4) = in, one of its defeaters, namely a4, is collectivelyundecided because [M(L)](a4) = undec.

in out undec

Agent 1-3 Agent 4-5 Agent 6-7

a3

a2

a1

a0

a4

Outcome

c2 c5

c0

c1

b4

b1

b2

b0

b3

c5c3c2

c0

c1

d

b4

b1

b2

b0

b3

a4

a1

a2

a0

a3

c4

d

c3

c7

c9

c8

c6

c2 c5

c0

c1

b4

b1

b2

b0

b3

c3

a3

a2

a1

a0

a4

c4c6c7

c9

c8

d

c4c6

c8

c9

c7

c2 c5

c0

c1

b4

b1

b2

b0

b3

c4

c3

c7

c9

c8

c6

a3

a2

a1

a0

a4

d

Fig. 12 A counterexample shows how, given semi-stable (respectively preferred) semantics,AWPR violates IN-CR2 and UNDEC-CR1.

30 Edmond Awad et al.

Remark 7 When agents can only vote for a semi-stable (respectively preferred)labelling, AWPR violates UNDEC-CR1. If an argument is collectively unde-cided, it is possible that one of its defeaters is collectively accepted.

Proof Suppose argument d has two defeaters, c4 and c5. Suppose we have7 agents, with votes as shown in Figure 12. Clearly, while d is collectivelyundecided because [M(L)](d) = undec, one of its defeaters, namely c4, iscollectively accepted because [M(L)](c4) = in.

Remark 8 When agents can only vote for a semi-stable (respectively preferred)labelling, AWPR violates UNDEC-CR2. If an argument is collectively unde-cided, it is possible that none of its defeaters is collectively undecided.

Proof Suppose argument c has two defeaters, a4 and b3. Suppose we have3 agents, with votes as shown in Figure 13. Clearly, while c is collectivelyundecided because [M(L)](c) = undec, none of its defeaters is collectivelyundecided.

b3

b2

in out undec

Agent 1

b1

b0

b4

a4

a1

a2

a0

a3

c

b4

b1

Agent 2

b2

b0

b3

a4

a1

a2

a0

a3

c

b3

b2

Agent 3

b1

b0

b4

a3

a2

a1

a0

a4

c

b3

b2

Outcome

b1

b0

b4

a4

a1

a2

a0

a3

c

Fig. 13 A counterexample shows how, given semi-stable (respectively preferred) semantics,AWPR violates UNDEC-CR2.

To sum up, the only restriction that would satisfy the Collective Rational-ity is the grounded semantics (Proposition 4). This is trivially true becauseonly one grounded labelling exists. However, stable semantics violates Collec-tive Rationality only because it violates OUT-CR. As for the semi-stable andpreferred semantics, they only satisfy IN-CR1, a property they inherit fromthe complete semantics. Refer to Table 1 for a summary of the results we havefound.

The results shown in Table 1 regarding the classical semantics are disap-pointing. However, possible future work may investigate for a set of graphswith some graph-theoretic properties, whether one semantics would be supe-rior to another. In other words, for any graph g ∈ G (where G is a set ofgraphs with some properties), if semantics S1 violates Collective Rationalitythen semantics S2 violates Collective Rationality. In this case, we might sayS1 is at least as good as S2 for the set of graphs G, from the point of view ofjudgment aggregation.

Judgment Aggregation in Multi-Agent Argumentation 31

SemanticsIN-CR

OUT-CRUNDEC-CR

IN-CR1 IN-CR2 UND-CR1 UND-CR2

GroundedYes Yes Yes Yes Yes

(Prop. 4) (Prop. 4) (Prop. 4) (Prop. 4) (Prop. 4)

StableYes Yes No Yes Yes

(Cor. 2) (Lem. 2) (Rem. 5) (Lem. 3) (Lem. 3)

Semi-stableYes No No No No

(Cor. 2) (Rem. 6) (Rem. 5) (Rem. 7) (Rem. 8)

PreferredYes No No No No

(Cor. 2) (Rem. 6) (Rem. 5) (Rem. 7) (Rem. 8)

CompleteYes No No No No

(Lem. 1) (Rem. 1) (Rem. 2) (Rem. 3) (Rem. 4)

Table 1 The Collective Rationality properties that are satisfied/violated by AWPR givendifferent semantics.

9 Restricting the Domain of Argumentation Graphs to SatisfyCollective Rationality

In an earlier section, we showed that, out of the ten defined postulates, AWPRonly violates Universal Domain and Collective Rationality. Then, we investi-gated whether AWPR can satisfy Universal Domain by introducing a tie-breaking rule. In Section 7, we characterised Collective Rationality in its con-stituents and, in Section 8, we checked how close AWPR is to satisfying them.In this section, we investigate whether AWPR can satisfy Collective Rationalityby restricting the argumentation framework to graphs with certain graph the-oretical properties. We show that graphs consisting of disconnected issues (anotion we define below) and graphs in which arguments have limited defeaters(in some sense) guarantee collectively rational outcomes when the AWPR isused.

9.1 Disconnected Issues

The notion of “issue” was defined in [7] in order to quantify disagreementbetween graph labellings. In this section, we use this notion to provide a pos-sibility result.

Crucial to the definition of the “issue” is the concept of “in-sync”. Twoarguments a and b are said to be in-sync if the (complete) label of one cannotbe changed without causing a change of equal magnitude to the label of theother.

Definition 8 (in-Sync ≡ [7]) Let Comp(AF ) be the set of all completelabellings for argumentation framework AF = 〈A,⇀〉. We say that two argu-ments a, b ∈ A are in-sync (a ≡ b):

a ≡ b iff (a ≡1 b ∨ a ≡2 b) (5)

where:

32 Edmond Awad et al.

– a ≡1 b iff ∀L ∈ Comp(AF ) : L(a) = L(b).– a ≡2 b iff ∀L ∈ Comp(AF ) : (L(a) = in ⇔ L(b) = out) ∧ (L(a) = out ⇔L(b) = in)

This relation forms an equivalence relation over the arguments, and theequivalence classes are called “issues”. An argumentation framework AF con-sists of a set of issues. One extreme case is when AF has only one issue (thatis all the arguments in graph are in-sync), the other extreme case is when AFcontains a set of issues each of which contains one argument (that is, no pairof arguments are in-sync).

Definition 9 (Issue [7]) Given the argumentation framework AF = 〈A,⇀〉,a set of arguments B ⊆ A is called an issue iff it forms an equivalence class ofthe relation in-Sync (≡).

For example, in Figure 14, the graph consists of three issues, namely {a1},{a2, a3}, and {a4, a5}.

a3 a2

a4

a1

a5

Fig. 14 An example about issues.

The following lemma is crucial in showing the main result of this subsection.We show that if the defeaters of an argument belong to the same issue as theargument, then the collective labelling of this argument chosen by AWPR isalways a legal labeling.

Lemma 4 Let AF = 〈A,⇀〉 be an argumentation framework. Let a ∈ A bean argument in this framework. If every defeater of a (call it b) belong to thesame issue of a (i.e. ∀b ∈ a−: b ≡ a), then AWPR would always produce alegal collective labelling for a.

Proof Let b1, . . . , bm ∈ A such that bj ∈ a− and a ≡ bj ∀j = 1, . . . ,m. Then,for every complete labelling L:

L(a) = out⇔ L(b1) = in⇔ . . .⇔ L(bm) = in (6a)

L(a) = in⇔ L(b1) = out⇔ . . .⇔ L(bm) = out (6b)

L(a) = undec⇔ L(b1) = undec⇔ . . .⇔ L(bm) = undec (6c)

From Equations 6a, 6b, and 6c, for every labelling profile L = (L1, . . . , Ln):

|{i : Li(a) = out}| = |{i : Li(b1) = in}| = . . . = |{i : Li(bm) = in}| (6d)

|{i : Li(a) = in}| = |{i : Li(b1) = out}| = . . . = |{i : Li(bm) = out}| (6e)

Judgment Aggregation in Multi-Agent Argumentation 33

|{i : Li(a) = undec}| = |{i : Li(b1) = undec}| = . . . = |{i : Li(bm) = undec}|(6f)

From Equations 6d, 6e, and 6f:

[M(L)](a) = out⇔ [M(L)](b1) = in⇔ . . .⇔ [M(L)](bm) = in (6g)

[M(L)](a) = in⇔ [M(L)](b1) = out⇔ . . .⇔ [M(L)](bm) = out (6h)

[M(L)](a) = undec⇔ [M(L)](b1) = undec⇔ . . .⇔ [M(L)](bm) = undec

(6i)From Equations 6g, 6h, and 6i, AWPR satisfies IN-CR, OUT-CR, and

UNDEC-CR with respect to a in this case. Then, a is always legally collectivelylabelled by AWPR if every defeater of it is in the same issue as a.

Given the previous lemma, we show that if the argumentation frameworkconsists of a set of disconnected issues, then AWPR satisfies Collective Ratio-nality for this framework.

Theorem 6 For every AF = 〈A,⇀〉 that consists of a set of disconnectedcomponents, each of which forms an issue, the argument-wise plurality rulewould always produce collectively rational outcomes.

Proof Since AF consists of a set of disconnected issues, then ∀a ∈ A, a has thefollowing property: ∀b ∈ A such that b ∈ a− then b ≡ a. From Lemma 4, a isalways legally collectively labelled by AWPR. Then AWPR satisfies CollectiveRationality for this AF .

9.2 Limited Defeaters

The notion of “Justification Status” is defined in [33]. Intuitively, the justifica-tion status of an argument is the set of possible labellings that this argumentcan take.

Definition 10 (Justification Status [33]) Let AF = 〈A,⇀〉 be an argu-mentation framework, and a ∈ A some argument. The justification status ofa is the outcome yielded by the function JS : A → 2{in,out,undec} such thatJS(a) = {L(a)|L ∈ Comp(AF )}.

There are six possible justification statuses. Neither ∅ nor {in, out} is a pos-sible justification statuses. The latter is shown in the following theorem:

Theorem 7 ([33, Theorem 2]) Let AF = 〈A,⇀〉 be an argumentationframework, and a ∈ A some argument. If AF has two complete labellings L1

and L2 such that L1(a) = in and L2(a) = out, then there exists a labelling L3

such that L3(a) = undec.

The following lemma shows that an argument with one of its defeatersbelong to the same issue as long as all the other defeaters of this argumenthave the justification status of {out}.

34 Edmond Awad et al.

Lemma 5 Let AF = 〈A,⇀〉 be an argumentation framework, and a, b ∈ Atwo arguments such that b ⇀ a. If the following holds:

∀c 6= b : (c ⇀ a⇒ JS(c) = {out}) (7a)

Then a and b belong to the same issue (i.e. a ≡ b). Moreover, a is alwayslegally collectively labelled by AWPR.

Proof One can show that:

L(a) = out⇔ L(b) = in (7b)

L(a) = in⇔ L(b) = out (7c)

L(a) = undec⇔ L(b) = undec (7d)

Hence, a ≡ b.Moreover, in a similar way to Lemma 4, one can show that, for every

possible profile L = (L1, . . . , Ln), the following holds:

– If [M(L)](a) = out then [M(L)](b) = in (b ∈ a−).– If [M(L)](a) = in then [M(L)](b) = out, and by Unanimity, ∀c 6= b : (c ⇀a⇒ [M(L)](c) = out).

– If [M(L)](a) = undec then [M(L)](b) = undec (b ∈ a−), and by Support-iveness, ∀c 6= b : (c ⇀ a⇒ [M(L)](c) 6= in).

Hence, a is always legally collectively labelled by AWPR.

Corollary 3 Let AF = 〈A,⇀〉 be an argumentation framework, and a ∈ Aan argument. If |a−| = 1 then a is always legally collectively labelled.

Proof From Lemma 5, a is always legally collectively labelled by AWPR.

Now we present the main theorem for this subsection. It says if all argu-ments have limited defeaters then AWPR always produces legally collectivelabelings. The limitation of the defeaters is characterised in both their numberand their justification statuses.

Theorem 8 Let AF = 〈A,⇀〉 be an argumentation framework. If AF doesnot contain an argument a ∈ A such that all the following hold:

1. a has at least two defeaters (i.e. |a−| ≥ 2),2. At least two defeaters’ justification statuses share undec (i.e. ∃b, c ∈ a−

s.t. b 6= c and undec ∈ JS(b) ∩ JS(c)).

then, AWPR satisfies Collective Rationality for this AF .

Proof Suppose, towards a contradiction, that AWPR violates Collective Ra-tionality for the AF described above. Then, there exists some argument a thatis illegally collectively labelled. Note that if |a−| = 0 (i.e. a has no defeaters),then a will be collectively labelled in (by Unanimity), and if |a−| = 1 (i.e. ahas only one defeater), then from Corollary 3, a would be legally collectivelylabelled. Hence, |a−| ≥ 2 (i.e. a has at least two defeaters).

Judgment Aggregation in Multi-Agent Argumentation 35

Since AWPR satisfies IN-CR1 by Lemma 1. Then, AWPR violates eitherIN-CR2, OUT-CR, UNDEC-CR1, or UNDEC-CR2. We will show that ineach of these cases, there exists two defeaters whose justification statuses shareundec:

– AWPR violates IN-CR2 i.e. [M(L)](a) = in and ∃b ∈ a− s.t. [M(L)](b) =undec. Then, by Supportiveness, undec ∈ JS(b). Since |a−| ≥ 2, then∃c 6= b s.t. c ∈ a−. Suppose ∀c 6= b: (c ∈ a− ⇒ undec /∈ JS(c)). Then, byTheorem 7, ∀c ∈ a− either JS(c) = {in} or JS(c) = {out}. However, if∃c ∈ a− s.t. JS(c) = {in} then JS(a) = {out}. By Unanimity, a wouldbe legally collectively labelled. The only left option is ∀c 6= b: (c ∈ a− ⇒JS(c)) = {out}. From Lemma 5, a would be legally collectively labelled.Contradiction. Then ∃c 6= b s.t. c ∈ a− and undec ∈ JS(c).

– AWPR violates OUT-CR i.e. [M(L)](a) = out and @b ∈ a− s.t. [M(L)](b) =in. We have three cases:– ∃b, c ∈ a− s.t. [M(L)](b) = [M(L)](c) = undec. Then, by Supportive-

ness, undec ∈ JS(b) ∩ JS(c).– ∃b ∈ a− s.t. [M(L)](b) = undec and ∀c ∈ a− [M(L)](c) = out. Then,

by Supportiveness, undec ∈ JS(b). Since |a−| ≥ 2, then ∃c 6= b s.t.c ∈ a−. Suppose ∀c 6= b: (c ∈ a− ⇒ undec /∈ JS(c)). Then, by Sup-portiveness and Theorem 7, the only left option is ∀c 6= b: (c ∈ a− ⇒JS(c) = {out}). From Lemma 5, a would be legally collectively labelled.Contradiction. Then ∃c 6= b s.t. c ∈ a− and undec ∈ JS(c).

– ∀b : (b ∈ a− ⇒ [M(L)](b) = out). Then, by Supportiveness, ∀b ∈ a−we have out ∈ JS(b). Since [M(L)](a) = out, by Supportiveness,out ∈ JS(a). Then, ∃b ∈ a− s.t. in ∈ JS(b). By Theorem 7, JS(b) ={in, out, undec}. Suppose ∀c 6= b: (c ∈ a− ⇒ undec /∈ JS(c)). In asimilar way as before we find that ∃c 6= b s.t. c ∈ a− and undec ∈JS(c).

– AWPR violates UNDEC-CR1 i.e. [M(L)](a) = undec and ∃b ∈ a− s.t.[M(L)](b) = in. By Supportiveness, in ∈ JS(b). Note that if JS(b) ={in} then JS(a) = {out}, then by Unanimity, [M(L)](a) = out. Contra-diction. Then, either JS(b) = {in, undec} or JS(b) = {in, out, undec}.Suppose ∀c 6= b: (c ∈ a− ⇒ undec /∈ JS(c)). In a similar way as before wefind that ∃c 6= b s.t. c ∈ a− and undec ∈ JS(c).

– AWPR violates UNDEC-CR2 i.e. [M(L)](a) = undec and ∀b : (b ∈ a− ⇒[M(L)](b) = out). By Supportiveness, undec ∈ JS(a). Then, ∃b ∈ a− s.t.undec ∈ JS(b). Suppose ∀c 6= b: (c ∈ a− ⇒ undec /∈ JS(c)). In a similarway as before we find that ∃c 6= b s.t. c ∈ a− and undec ∈ JS(c).

Note that the condition given on AF in Theorem 8 is neither a generalisa-tion nor a special case of the condition on AF in Theorem 6. Example 7 showsan AF that satisfies the condition in Theorem 6 (i.e. disconnected issues), butviolates the condition in Theorem 8 (i.e. limited defeaters), while Example 8shows an AF that satisfies the condition in Theorem 8 (i.e. limited defeaters),but violates the condition in Theorem 6 (i.e. disconnected issues).

36 Edmond Awad et al.

Example 7 Note that the argumentation framework in Figure 15 satisfies thecondition in Theorem 6. All the arguments a, b, c, d, and e are in the sameissue, so this AF consists of disconnected issues (only one issue in this case).However, this AF violates the condition in Theorem 8, since argument a isdefeated by two arguments b and c, each of these defeaters has a justificationstatus of {in, out, undec}, and so their justification statuses share undec.

e

b

c

ad

Fig. 15 An example for an AF that satisfies the condition in Theorem 6, but violates thecondition in Theorem 8.

Example 8 Note that the argumentation framework in Figure 16 satisfies thecondition in Theorem 8. The only argument that is defeated by more than oneargument is argument a which has two defeaters b and c. Moreover, undec /∈JS(c), so undec /∈ JS(b)∩JS(c). However, this AF violates the condition inTheorem 6, since it contains two connected issues. The first issue is {a, b, d}and the second issue is {c, e}.

d b

c

a

e

Fig. 16 An example for an AF that satisfies the condition in Theorem 8, but violates thecondition in Theorem 6.

10 Discussion & Conclusion

In this paper, we presented an extensive analysis of social-choice-theoreticaspects of Dung’s highly-influential argumentation semantics. We introducedand explored the following question:

Given a set of agents, each with a specific subjective evaluation (i.e.labelling) of a given set of conflicting arguments, how can agents reacha collective decision on how to evaluate those arguments?

In this context, the specific original contributions of this paper can besummarised as follows:

Judgment Aggregation in Multi-Agent Argumentation 37

1. Defining the multi-agent collective argument evaluation problem, includ-ing adapting various desirable postulates from the classical judgment andpreference aggregation literature.

2. Analysing the “argument-wise plurality rule”, proving many of its key prop-erties, showing its failure to satisfy “Universal Domain” and “CollectiveRationality”.

3. Proving general impossibility results on the existence of any argument eval-uation aggregation operator that satisfies some minimal properties. Thenfully characterizing conditions under which an aggregation operator doessatisfy Collective Rationality.

4. Analysing the argument-wise plurality rule with respect to the charac-terised conditions.

5. Showing that using other classical semantics as a domain restriction for theargument-wise plurality rule is not enough (except for the grounded seman-tics) for the fulfillment of the characterised conditions—and characterisingexactly where each semantics fails.

6. Showing graph theoretic restrictions on the argumentation framework wouldhelp argument-wise plurality rule to satisfy “Collective Rationality”.

Argumentation-based semantics have mainly been compared on the basisof how they deal with specific benchmark problems that reflect specific logicalstructures from the point of view of a single omniscient observer (e.g. argumentgraph structures with odd-cycles, floating defeaters etc.). Recently, it has beenargued that argumentation semantics must be evaluated based on more generalintuitive principles [3]. Our work can be seen as a contribution in this direction,focusing on issues relating to multi-agent preferences.

The closest work to the present paper is Caminada and Pigozzi [10]. Intheir work, they propose three aggregation operators, namely sceptical, cred-ulous and super credulous. Although the operators satisfy Collective Ratio-nality, they violate Independence. These operators are also more applicableto scenarios with small numbers of agents where individuals are expected topublicly defend the group decision (e.g. a jury). Argument-wise plurality rule,on the other hand, can be applied to classical scenarios with a large groupof agents where some individuals might naturally disagree with the opinionof the group. Additionally, unlike our work, their work focuses on the pro-posed operators with only little attention to the general aggregation problem.Only four postulates are proposed, namely Universal Domain, Collective Ra-tionality, Anonymity, and Independence, and there are no general impossibilityresults that holds for any operator.

The social choice theoretic Arrovian properties have been analyzed in thecontext of social argument justification in [31]. An extended argumentationframework AFn = 〈A, ⇀1, . . . ,⇀n〉 is defined, where each ⇀i, 1 ≤ i ≤ n, isa particular attack relation among the arguments in A, representing differentattack criteria. Then, the authors define an aggregate argumentation frame-work AF ∗ = 〈A, F(⇀1, . . . ,⇀n)〉, where F(⇀1, . . . ,⇀n) is an attack relationobtained by the aggregation of the individual attack criteria ⇀1, . . . ,⇀n, via

38 Edmond Awad et al.

different kinds of mechanisms (e.g. majority voting, qualified voting and mech-anisms that can be described by classes of decisive sets). The aggregation ofindividual attack criteria can not be assimilated to the kind of mechanisms pro-posed here. In [31] an individual may sanction an attack between two given ar-guments while another individual may not, which in terms of labellings meansthat for the same pair of arguments there may exist two possible labellings, say(in, out) and (in, in). This is impossible in our setting. Hence, the Arrovianproperties (e.g. Collective Rationality) are conceived in an essentially differentway.

In [6] the authors analyse the problem of aggregating different individualargumentation frameworks over a common set of arguments in order to obtaina unique socially justified set of arguments. One of the procedures consideredthere is one in which each individual proposes a set of justified arguments andthen the aggregation leads to a unique set of socially justified arguments. Theargument-wise plurality rule mechanism proposed here fits this procedure forthe special case in which individually justified arguments are simply the setsof arguments labelled in for each individual.

There is much work on using an individual agent’s preferences to help eval-uate arguments (e.g. based on given priorities over rules [25]). But this line ofwork does not address the preferences of multiple agents and how they may beaggregated. In other related work, Bench-Capon [4] associates arguments withvalues they promote or demote, and considers different audiences with differentpreferences over those values. Such preferences determine whether particulardefeats among arguments succeed. Thus, one gets different argument graphs,one for each audience. Bench-Capon uses this to distinguish between an ar-gument’s subjective acceptance with respect to a particular audience, and itsobjective acceptance in case it is acceptable with respect to all possible au-diences. Our work differs in two important ways. Firstly, in our framework,an agent (or equivalently, an audience) does not have preferences over indi-vidual arguments, but rather preferences over how to evaluate all argumentscollectively (i.e. over labellings). Secondly, our concern here is not with howindividual agents (or audiences) accept an argument, but rather on the pos-sibility of achieving important social-choice properties in the final aggregatedargument labelling.

In relation to aggregation, Coste-Marquis et al explored the problem ofaggregating multiple argumentation frameworks [11]. Each agent’s judgmentconsists of a different argument graph altogether. This contrasts significantlywith our work, in which agents do not dispute the argument graph, but ratherhow it must be evaluated/labelled. Our setting is more akin to a jury situation,in which all arguments have been presented by the prosecution and defenseteam, and are visible to the jury members. The jury members themselves donot introduce new arguments, but are tasked with aggregating their individualjugdgments about the arguments presented to them.

Finally, we refer to the work of Rahwan and Larson [26] on strategic be-haviour when arguments are distributed among agents, and where these agentsmay choose to show or hide arguments. Thus, their interest is in how agents

Judgment Aggregation in Multi-Agent Argumentation 39

contribute to the construction of the argument graph itself, which is then eval-uated centrally by the mechanism (e.g. a judge). In contrast, our present paperis concerned with how agents individually cast votes on how to evaluate eachargument in a given fixed graph.

Our results on the aggregation of different argument evaluations by mul-tiple agents provide a new approach for conflict-resolution in multi-agent sys-tems. While much research has focused on the aggregation of simpler pref-erences (e.g. binary preference statements expressing preorders) and its as-sociated paradoxes, our research focuses on the aggregation of more complexpreferences, expressed as argumentative points of view.

Our work also opens new research problems for the computational socialchoice community. As is the case with other aggregation domains, the aggrega-tion paradox in argument evaluation is an example of a fundamental barrier.Thus the impossibility results are important because they give conclusive an-swers and focus research in more constructive directions (e.g. weakening thedesired properties in order to avoid the paradox). Another promising researchdirection is characterising the computational complexity of checking whethera given argumentation graph would violate desirable properties like collectiverationality. An algorithmic agenda would complement this research by pro-viding efficient algorithms for such problems. Strategic manipulation, by mis-reporting one’s true vote, is also an important area of investigation, especiallywhen such manipulations are exercised by coalitions of agents.

Acknowledgements We are grateful to Andrew Clausen for his insightful comments. Weare especially grateful to Martin Caminada for discussions surrounding the second impos-sibility result. Much of this paper was written during a visit of Edmond Awad to the Uni-versity of Luxembourg which was generously supported by SINTELNET. Richard Booth issupported by the FNR (DYNGBaT project).

References

1. Arrow, K.J.: Social choice and individual values. Wiley, New York NY, USA (1951)2. Arrow, K.J., Sen, A.K., Suzumura, K. (eds.): Handbook of Social Choice and Welfare,

vol. 1. Elsevier Science Publishers (North-Holland) (2002)3. Baroni, P., Giacomin, M.: On principle-based evaluation of extension-based argu-

mentation semantics. Artificial Intelligence 171(10–15), 675–700 (2007). DOI doi:10.1016/j.artint.2007.04.004

4. Bench-Capon, T.J.M.: Persuasion in practical argument using value-based argumenta-tion frameworks. Journal of Logic & Computation 13(3), 429–448 (2003)

5. Bench-Capon, T.J.M., Dunne, P.E.: Argumentation in artificial intelligence. ArtificialIntelligence 171(10–15), 619–641 (2007)

6. Bodanza, G., Auday, M.: Social argument justification: some mechanisms and conditionsfor their coincidence. In: C. Sossai, G. Chemello (eds.) Symbolic and QuantitativeApproaches to Reasoning with Uncertainty, 10th European Conference, ECSQARU,Lecture Notes in Computer Science, vol. 5590, pp. 95–106. Springer (2009)

7. Booth, R., Caminada, M., Podlaszewski, M., Rahwan, I.: Quantifying disagreementin argument-based reasoning. In: Proceedings of the 11th International Conferenceon Autonomous Agents and Multiagent Systems-Volume 1, pp. 493–500. InternationalFoundation for Autonomous Agents and Multiagent Systems (2012)

40 Edmond Awad et al.

8. Caminada, M.: On the issue of reinstatement in argumentation. In: M. Fisher,W. van der Hoek, B. Konev, A. Lisitsa (eds.) Proceedings of the 10th European Confer-ence on Logics in Artificial Intelligence (JELIA), Lecture Notes in Computer Science,vol. 4160, pp. 111–123. Springer (2006)

9. Caminada, M., Gabbay, D.M.: A logical account of formal argumentation. Studia Logica93(2-3), 109–145 (2009)

10. Caminada, M., Pigozzi, G.: On judgment aggregation in abstract argumentation. Au-tonomous Agents and Multi-Agent Systems 22(1), 64–102 (2011)

11. Coste-Marquis, S., Devred, C., Konieczny, S., Lagasquie-Schiex, M.C., Marquis, P.: Onthe merging of Dung’s argumentation systems. Artificial Intelligence 171(10–15), 730–753 (2007)

12. Dietrich, F.: A generalised model of judgment aggregation. Social Choice and Welfare28, 529–565 (2007)

13. Dokow, E., Holzman, R.: Aggregation of binary evaluations with abstentions. Journalof Economic Theory 145(2), 544–561 (2010)

14. Dokow, E., Holzman, R.: Aggregation of non-binary evaluations. Advances in AppliedMathematics 45(4), 487–504 (2010)

15. Dung, P.M.: On the acceptability of arguments and its fundamental role in nonmono-tonic reasoning, logic programming and n-person games. Artificial Intelligence 77(2),321–358 (1995)

16. Gartner, W.: A Primer on Social Choice Theory. Oxford University Press (2006)17. Gibbard, A.: Manipulation of voting schemes. Econometrica 41, 587–601 (1973)18. Grandi, U.: Binary aggregation with integrity constraints. Ph.D. thesis (2012)19. Grossi, D., Pigozzi, G.: Introduction to judgment aggregation. In: N. Bezhanishvili,

V. Goranko (eds.) Lectures on Logic and Computation. ESSLLI’10 and ESSLLI’11,Selected Lecture Notes, vol. 7388, pp. 160–209

20. List, C.: The theory of judgment aggregation: An introductory review. Synthese 187(1),179–207 (2012)

21. List, C., Pettit, P.: Aggregating sets of judgments: An impossibility result. Economicsand Philosophy 18, 89–110 (2002)

22. List, C., Polak, B.: Introduction to judgment aggregation. Journal of economic theory145(2), 441–466 (2010)

23. List, C., Puppe, C.: Judgment aggregation: a survey. In: P. Anand, C. Puppe, P. Pat-tanaik (eds.) The handbook of rational and social choice. Oxford University Press,Oxford, UK (2009)

24. Muller, E., Satterthwaite, M.A.: The equivalence of strong positive association andstrategy-proofness. Journal of Economic Theory 14(2), 412–418 (1977)

25. Prakken, H., Sartor, G.: Argument-based logic programming with defeasible priorities.Journal of Applied Non-classical Logics 7, 25–75 (1997)

26. Rahwan, I., Larson, K.: Mechanism design for abstract argumentation. In: L. Padgham,D. Parkes, J. Mueller, S. Parsons (eds.) 7th International Joint Conference on Au-tonomous Agents & Multi Agent Systems, AAMAS’2008, Estoril, Portugal, pp. 1031–1038 (2008)

27. Rahwan, I., Simari, G.R. (eds.): Argumentation in Artificial Intelligence. Springer (2009)28. Rahwan, I., Tohme, F.: Collective argument evaluation as judgement aggregation. In:

Proceedings of the 9th International Conference on Autonomous Agents and MultiagentSystems: volume 1-Volume 1, pp. 417–424. International Foundation for AutonomousAgents and Multiagent Systems (2010)

29. Satterthwaite, M.A.: Strategy-proofness and Arrow’s conditions: Existence and cor-respondence theorems for voting procedures and social welfare functions. Journal ofEconomic Theory 10, 187–217 (1975)

30. Sen, A.: The impossibility of a paretian liberal. The journal of political economy 78(1),152–157 (1970)

31. Tohme, F., Bodanza, G., Simari, G.: Aggregation of attack relations: a social-choicetheoretical analysis of defeasibility criteria. In: S. Hartmann, G. Kern-Isberner (eds.)Foundations of Information and Knowledge Systems, 5th International Symposium,FoIKS, Lecture Notes in Computer Science, vol. 4932, pp. 8–23. Springer Verlag, Berlin,Germany (2008)

Judgment Aggregation in Multi-Agent Argumentation 41

32. Vincke, P.: Aggregation of preferences: a review. European Journal of OperationalResearch 9(1), 17–22 (1982)

33. Wu, Y., Caminada, M., Podlaszewski, M.: A labelling-based justification status of ar-guments. Studies in Logic 3(4), 12–29 (2010)