European Journal of Operational Research 207 (2010) 1599–1607

Contents lists available at ScienceDirect

European Journal of Operational Research

journal homepage: www.elsevier .com/locate /e jor

Innovative Applications of O.R.

Performance appraisal based on distance function methods

Rocío de Andrés a, José Luis García-Lapresta b,*, Jacinto González-Pachón c

a PRESAD Research Group, Dep. de Fundamentos del Análisis Económico e H.I.E., Universidad de Valladolid, 47011 Valladolid, Spainb PRESAD Research Group, Dep. de Economía Aplicada, Universidad de Valladolid, 47011 Valladolid, Spainc Dep. de Inteligencia Artificial, Universidad Politécnica de Madrid, 28660 Boadilla del Monte, Madrid, Spain

a r t i c l e i n f o a b s t r a c t

Article history:Received 5 May 2009Accepted 11 June 2010Available online 18 June 2010

Keywords:Performance appraisalMulti-criteria decision makingFinite scalesExtended Goal Programming approach

0377-2217/$ - see front matter � 2010 Elsevier B.V. Adoi:10.1016/j.ejor.2010.06.012

* Corresponding author. Tel.: +34 983 184 391.E-mail address: lapresta@eco.uva.es (J.L. García-La

Performance appraisal is a process used by some firms to evaluate their employees’ efficiency and pro-ductivity in order to plan their promotion policy, salary policy, layoffs policy, etc. Initially this processwas just carried out by the executive staff, but recently it has evolved into an evaluation process basedon the opinion of different reviewers, supervisors, collaborators, customers and the employees them-selves (360-degree method). In such an evaluation process the reviewers evaluate some indicatorsrelated to employees performance appraisal. In this paper we propose an evaluation framework wherethere are different sets of reviewers taking part in the evaluation process. Since reviewers have a differentknowledge about the evaluated employee, it seems suitable to offer a flexible framework in which differ-ent reviewers can express their assessments in different finite scales according to their knowledge. Thefinal aim is to compute a global evaluation for each employee, that can be used by the management teamto make their decisions regarding their human resources policy. In this way, to obtain a global evaluationfor each employee, we propose a methodology able to aggregate individual valuation in a metric Lp

framework. In this context, the associated optimization problems can be reduced to an Extended GoalProgramming formulation that is very easy to compute.

� 2010 Elsevier B.V. All rights reserved.

1. Introduction

Throughout history, organizations and companies have beenadapting themselves to established conditions to get their survivaland success. Nowadays, global competition affects most companiesand organizations. The principal organizations’ aim is to remaincompetitive in this context. To adapt to changes, companies devel-op strategies that let them be more efficient in each new context.In this way, those strategies must be able to mould each one ofcompanies’ components to adapt correctly to changes. Presently,Human Resources are a fundamental company’s component. In thisway, all strategic efforts of companies have to be headed for thedevelopment of new methodologies by adapting human resourcesto the new requirements of companies.

Little by little organizations and companies have been introduc-ing different methods of performance appraisal to reach such sur-vival and success (see Banks and Roberson (1985), Bretz et al.(1992) and Miner (1998) among others). One of the objectives ofthe Human Resources Department consists in obtaining effectiveperformance appraisal systems. Thus, companies carry out perfor-mance appraisal processes taking into account the principal ele-ments that affect workers in a direct or in an indirect way.

ll rights reserved.

presta).

Through an effective performance appraisal system, companiescan realize the following functions (see Cleveland et al. (1989)and Fisher et al. (2006)):

� Developmental uses: to measure performance goals, to relocateemployees for developmental purposes, to identify employeestraining needs, etc.� Administrative uses: salary, promotion, retention or termina-

tion, layoffs, discipline, etc.� Organizational maintenance: human resource planning, to deter-

mine organization training needs, to evaluate organizational goalachievement, to evaluate human resource systems, etc.� Documentation: document human resource decisions and help

meet legal requirements.

Many companies tend to use informal methods, where onlysupervisors evaluate employees and often develop formal perfor-mance but only used by supervisors to evaluate employees. Dueto this fact, the results of these kinds of evaluation processes areoften biased and subjective, can present the halo effect (cognitiveturn which makes to think that some limited characteristics areapplied to everything), can be based on prejudice (about religion,race, accent, gender, nationality, etc.), etc. (see Latham and Wexley(1981), Kerr (1985), Frunzi and Savini (1996) and Fisher et al.(2006), among others).

1600 R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607

There is another kind of method which uses information frommany people who can truly respond to know how an employeeperforms on the job. The so-called 360-degree appraisal or integralevaluation is a mechanism for evaluating worker’s performancebased on judgment from everyone with whom the employeecomes in contact: supervisors, collaborators, colleagues, customersand oneself included (see Edwards and Ewen (1996)). The 360-degree appraisal system (see Fig. 1) presents some advantages withregard to the traditional systems and it can be emphasized (seeBanks and Roberson (1985), Edwards and Ewen (1996), Baronand Kreps (1999), Jarman (1999), Fletcher (2001), Pfau and Kay(2002), among others):

� It is more extensive due to the fact that integral evaluation col-lects information from different points of view.� It can reduce the bias and prejudice because the information

comes from several people, not just one.� It encourages the Human Resources Department to establish

policies of internal selection more clearly based on the resultsof the evaluation process.� From this evaluation system the Human Resources Department

can define training and development plans for employees basedon individual and group performance appraisal results.� It allows companies to identify successful people more easily

with the aim of reinforcing, recognizing and encouraging theirresults.

In order to overcome the drawbacks associated with the tradi-tional performance appraisal process and to adopt the advantagesof the integral evaluation, in this paper, we propose a 360-degreeappraisal model where different sets of reviewers have to evaluateemployees attending to different criteria and attributes. Takinginto account that the different groups of reviewers may have a dif-ferent degree of knowledge about the evaluated employees, wethen suggest a flexible evaluation framework where the appraiserscan express their assessments in different scales according to theirknowledge and expertise; such scales can come from a quantitativeor qualitative nature (see Baron and Kreps (1999)). In the litera-ture, it is possible to find performance appraisal processes basedon quantitative information and/or on qualitative informationdepending on the kind of criterion to be evaluated, although mostof them represent such information in a numerical way. In this pa-per, we represent the information from the reviewers in a numer-ical way with the purpose of simplifying the computing task.Nevertheless, the information may be represented by means of lin-guistic variables following the methodology proposed in de Andréset al. (2010).

Although the aggregation of the information in performanceappraisal is an essential process, most of the companies which usu-ally carry it out by use of weighted averages without taking into

Colaborators Co-workersEmployee

Customers

Supervisors

Fig. 1. 360-degree appraisal.

account whether this type of aggregation operator has or has notgot good properties. Our proposal is to analyze the aggregationproblem from a different point of view by means of the use of aLp metric framework where every associated optimization problemcan be formulated as a Goal Programming problem that is veryeasy to compute. For this objective, we follow the approach pro-posed by González-Pachón and Romero (2006, 2007) for aggregat-ing individual evaluations because this method allows us to carryout the aggregation procedure from two different and, sometimes,incompatible social principles: the government of the majority andthe consideration to minorities. Moreover, to consider sensitivitywith respect to extreme disagreement evaluations, alternativesolutions may be obtained through convex combinations of theprevious points of view.

The paper is organized as follows. Section 2 includes the frame-work of our integral evaluation proposal. Section 3 is devoted toexplain the unification information phase, the aggregation phaseand exploitation phase in our model. Next, in Section 4 we proposean illustrative example. Finally, some concluding remarks are in-cluded in Section 5.

2. Integral evaluation framework

In this section, the framework of our proposal for an integralevaluation model is introduced.

In order to show how a company could carry out an integralevaluation process, let us suppose a company which wants to eval-uate the performance of their employees. It is supposed there is aset of employees X = {x1, . . . ,xn} to be evaluated by the followingcollectives:

� A set of supervisors or the management team: A = {a1 , . . . ,ar}.� A set of colleagues, partners or co-workers: B = {b1 , . . . ,bs}.� A set of customers and/or subordinates: C = {c1 , . . . ,ct}.

We consider several criteria Y1, . . . , Yq for evaluating each em-ployee. We notice there is a general set of criteria to evaluateemployees but each collective of reviewers could only assessemployees attending to a number of them following the guidelinesestablished by the Human Resources Department.

By means of aikj ; b

ikj and cik

j we will denote the assessments ofai 2 A, bi 2 B and ci 2 C on the candidate xj according to the compe-tency Yk, respectively. Therefore, there are at most (r + s + t)qassessments for each employee provided by the three collectives.

As we have previously mentioned, the members of the differentcollectives usually have a different degree of knowledge aboutevaluated employees. So, we assume that each collective can usedifferent finite scale of real numbers to express their assessmentsto each employee for each criterion Yk:

� Supervisors

SkA ¼ fsAk

1 ; . . . ; sAkf g � R; sAk

1 < � � � < sAkf :

� Collaborators

SkB ¼ fsBk

1 ; . . . ; sBkg g � R; sBk

1 < � � � < sBkg :

� Customers and/or subordinates

SkC ¼ fsCk

1 ; . . . ; sCkh g � R; sCk

1 < � � � < sCkh :

It takes notice of each term of a numerical scale that can be de-scribed by a linguistic label following Likert’s methodology (seeLikert (1932)).

In the literature (see Antes et al. (1998), Bouyssou et al. (2000),Arfi (2005) and Martínez (2007)), the use of the decision analysis to

R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607 1601

accomplish evaluation processes has got good results. Therefore, inthis paper we shall propose a performance appraisal process basedon a classical decision analysis approach (see Clemen (1995) andMartínez (2007)) with three phases (see Fig. 3):

(1) Normalization information phase: Our proposal considers thatreviewers can express their opinions about employees in dif-ferent scales of real numbers according to their knowledge.Therefore and before carrying out the aggregation process,it is necessary to standardize or normalize all gathered infor-mation into a unique domain, the interval [0,1].

(2) Aggregation phase: Once the normalization phase has beenachieved, the aggregation process can start. Our aggregationprocess is based on the distance aggregation method pro-posed by González-Pachón and Romero (1999) and Romero(2001). In this way and to obtain a final value for eachemployee xj is carried out an aggregation procedure withthree different steps (in this moment we do not pay atten-tion to the way we aggregate the information; it is includedin Section 3). These steps consist of (see Fig. 2):

(a) Computing collective criteria values. The aggregation pro-

cess is initiated by obtaining a collective value for eachemployee xj taking into account the assessments of everycollective for each criterion Yk.

Aggregation process

Global criteria values

Global values

Collective criteria values

Supervisors Collaborators

Customers Employees

Fig. 2. Steps of the aggregation process.

Fig. 3. Decision scheme for integral ev

(b) Computing global criteria values. In the second stage weaggregate the previous reviewers’ collective valuesproviding a global value for each criterion to everyemployee.

(c) Computing global value. Finally, is obtained a globalassessment for each employee aggregating the previousglobal criteria values.

All outcomes can be used for sorting and ranking employees inorder to establish the human resources policy.

(3) Rating phase. In the exploitation phase, companies are goingto classify and order employees x1, . . . ,xn to adhere to theHuman Resources policy.

Some models of performance appraisal include the opinion ofeach employee about her/himself. Although the opinion of eachemployee about her/himself can be useful for the organization,companies do not take into account this information in the aggre-gation process because including the employees’ self-evaluationcould disturb the aggregation phase, that means the correspondingoutcomes could be biased by self-evaluations. However, opinionsof employees about themselves are used by companies in the rat-ing phase with the purpose of improving their performance andsynchronizing companies goals with employees goals (see Fig. 3).This part of performance appraisal procedure is so-called feedbackand coaching process (see Frunzi and Savini (1996) and Fisher et al.(2006)).

3. Distance-based aggregation procedure for performanceappraisal

In this section we carry out in detail the different phases of per-formance appraisal process mentioned above.

3.1. The normalization phase

In order to aggregate properly the assessments from reviewers,all gathered information should be unified. Therefore, all informa-tion is standardized in the interval [0,1] in the normalizationphase.

� For supervisors. Taking into account the scale used by supervi-sors to evaluate each employee for the criterion Yk, the normal-ized previous scale is defined as:

SkA ¼ fsAk

1 ; . . . ; sAkf g � ½0;1�;

where

aluation process.

1602 R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607

�sAkm ¼

sAkm � sAk

1

sAkf � sAk

1

2 ½0;1�; m ¼ 1; . . . ; f :

Similarly we can standardize the individual assessments in theinterval [0,1] for the remaining collectives.� For collaborators. Sk

B ¼ �sBk1 ; . . . ;�sBk

g

n o� ½0;1�, where �sBk

m ¼sBk

m �sBk1

sBkg �sBk

12

½0;1�; m ¼ 1; . . . ; g.� For customers and/or subordinates. Sk

C ¼ �sCk1 ; . . . ;�sCk

h

� �� ½0;1�,

where �sCkm ¼

sCkm �sCk

1sCk

h�sCk

12 ½0;1�; m ¼ 1; . . . ;h.

3.2. The aggregation phase

One of the aims of the aggregation phase is to obtain a valuethat assesses the performance of the evaluated worker accordingto the different collectives that have evaluated her/him. To do that,the assessments provided by the members of different collectiveswill be aggregated. The aggregation procedure mentioned inSection 2 can be developed through several methodologies.

Decisional problems with multiple attributes, decision variablesand restrictions are not easy to be solved by means of classical mul-ti-objective optimization methods. For this reason it seems suitableto use another type of more flexible methodologies, inside of thesemethodology lines Goal Programming (GP) can be included. GP tech-niques are not based on the objectives optimization but are basedon ‘‘satisficing” philosophy (see Simon (1957)). This methodology af-firms that companies decisional problems lack complete informa-tion, have more than one manifold objective (that in additionthey present disagreement among them), limited resources, etc.

In this paper we propose an operational method to aggregateinformation in a distance-based framework by a Goal Program-ming formulation, see González-Pachón and Romero (1999, 2001).

In order to present the aggregation procedure, we consider y* tobe the target value from the individual assessments y1,. . .,ym,

Aggðy1; . . . ; ymÞ ¼ y�:

The problem is how to determine y*. This target value representsthe unknown value of our aggregation problem. It can be obtainedby formulating a distance function model as a measure of disagree-ment between the individual assessments y1, . . . ,ym and the desiredtarget value y*. Then, we deal with the following problem formu-lated for the Minkowski metric Lp (see Yu (1973)), for p P 1:

min d ðy1; . . . ; ymÞ; ðy�; . . . ; y�Þ� �

¼minXm

i¼1

wpi jy

i � y�jp !1=p

;

where wi is the parameter attached to the discrepancy between theachievement of the i-th criterion and the target value.

The resolution of this problem is not easy because of the non-linear and non-differentiable character of the function. In orderto avoid some of the above disadvantages in the preceding func-tion, the previous problem can be transformed into the followingGoal Programming problem (see Romero (1991), González-Pachónand Romero (1999), Dopazo and González-Pachón (2003) and Gon-zález-Pachón and Romero (2006)):

minXm

i¼1

wiðgi þ qiÞð Þp

s:t: y� þ g1 � q1 ¼ y1;

. . .

y� þ gm � qm ¼ ym;

0 6 y� 6 1;g1 P 0; . . . ;gm P 0;q1 P 0; . . . ;qm P 0;

ð1Þ

where gi and qi are the negative and positive deviation variables,respectively that measure the difference among the target valueand the individual assessments.

Notice that different target values y* are obtained for differentmetrics Lp. We also note that:

� If we consider low values of p, then the problem pays moreattention to medium values.� If we consider high values of p, then the problem pays

more attention to the extreme values. If we consider thelimit case p =1, this converts the Goal Programming probleminto the following MINIMAX (Chebyshev) formulation, wherethe disagreement of the most displaced assessments is mini-mized (see for instance Ignizio and Caliver (1994) and Romero(2001)):

min D

s:t: w1ðg1 þ q1Þ 6 D;

. . .

wmðgm þ qmÞ 6 D;

y� þ g1 � q1 ¼ y1;

. . .

y� þ gm � qm ¼ ym;

0 6 y� 6 1;

g1 P 0; . . . ;gm P 0;

q1 P 0; . . . ;qm P 0:

ð2Þ

where D represents the disagreement of the social group or individ-ual with the opinions more displaced (i.e., more significantly differ-ent) from the consensus obtained.

One of the main problems in this kind of multi-criteria prob-lems is to select the right value of p which captures the essentialcharacteristics of real problems. Romero (2001) proposes a generalGoal Programming structure called Extended Goal Programmingwhich includes cases (1) and (2):

min ð1� kÞDþ kXm

i¼1

wiðgi þ qiÞð Þ

s:t: w1ðg1 þ q1Þ 6 D;

. . .

wmðgm þ qmÞ 6 D;

y� þ g1 � q1 ¼ y1;

. . .

y� þ gm � qm ¼ ym;

0 6 y� 6 1;

g1 P 0; . . . ;gm P 0;

q1 P 0; . . . ;qm P 0;

ð3Þ

where the value of k 2 [0,1] states the balance between the minimi-zation of the maximum lack of success and the minimization of theweighted sum of the deviation variables in relation to the targetvalues.

González-Pachón and Romero (2006, 2007) considered the Ex-tended Goal Programming formulation to propose a method foraggregating individual valuations. In their formulation, the pparameter is interpreted as a compensatory parameter betweentwo desirable social principles in all consensus process: the gov-ernment of the majority, p = 1, and the consideration to minorities,p =1. Thus, as p 2 [1,1] increases, more importance is given tothe biggest disagreements valuations (see Yu (1973)). To considersensitivity with respect to extreme disagreement valuations, analternative solution may be obtained through a convex combina-

R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607 1603

tion of metric Lp for p = 1 and p =1 using a value for k � parame-ter. For k = 1, the consensus valuation is defined by minimizingmetric Lp for p = 1 (principle of majority). For k = 0, the consensusvaluation is defined by minimizing metric Lp for p =1 (principleof minority). For control parameter k values belonging to the openinterval (0,1), compromise consensus valuations (if they exist)could be obtained.

3.2.1. The aggregation stepsWe now show how to aggregate the information provided by

reviewers following the steps presented in Section 2. For this pur-pose, we consider the Extended Goal Programming formulationthat appeared previously.

� Collective criteria values. For each employee xj, criterion Yk andset of reviewers, we aggregate the individual assessments.– For supervisors:

vkAðxjÞ ¼ Akk

A a1kj ; . . . ; ark

j

� �¼ a�kj 2 ½0;1�;

by solving the following problem

min ð1� kkAÞDþ kk

A

Xr

i¼1

wAki ðgji þ qjiÞ

� �

s:t: wAk1 ðgj1 þ qj1Þ 6 D;

. . .

wAkr ðgjr þ qjrÞ 6 D;

a�kj þ gj1 � qj1 ¼ a1kj ;

. . .

a�kj þ gjr � qjr ¼ arkj ;

0 6 a�kj 6 1;

gj1 P 0; . . . ;gjr P 0

qji P 0; . . . ;qjr P 0:

Similarly we can aggregate the individual assessments for theremaining collectives.

– For collaborators:

Table 1Collective values for Yk.

Reviewers Reviewers’ collective values

vkBðxjÞ ¼ Akk

B b1kj ; . . . ; bsk

j

� �¼ b�kj 2 ½0;1�;

– For customers and/or subordinates:

vkCðxjÞ ¼ Akk

C c1kj ; . . . ; ctk

j

� �¼ c�kj 2 ½0;1�:

Although the aggregation process is similar in every set of review-ers, it is possible to consider different values of w�i for each review-ers’ collective and for each criterion. Generally, the weightingvectors used in the aggregation process are fixed by the HumanResources Department according to the goals established in theirhuman resources policy.In the same way, different values of k�� for each group of reviewersand each criterion, kk

A; kkB and kk

C , can be considered. If companieswant to obtain an equitable aggregation process, the values of k��should be near 0, but if they want to obtain an efficient aggregationprocess, the values of k�� should be near 1. Thus, for several valuesof w�i and k�� we will obtain different points of view to get the eval-uation for each employee xj.� Global criteria values.

Supervisors a�k1 ; . . . ; a�kn

Colleagues b�k1 ; . . . ; b�kn

Customers c�k1 ; . . . ; c�kn

vkðxjÞ ¼ Akk

a�kj ; b�kj ; c

�kj

� �¼ x�kj 2 ½0;1�;

by solving the following problem

min ð1� kkÞDþ kkX3

i¼1

wki ðgji þ qjiÞ

� �

s:t: wk1ðgj1 þ qj1Þ 6 D;

wk2ðgj2 þ qj2Þ 6 D;

wk3ðgj3 þ qj3Þ 6 D;

x�kj þ gj1 � qj1 ¼ a�kj ;

x�kj þ gj2 � qj2 ¼ b�kj ;

x�kj þ gj3 � qj3 ¼ c�kj ;

0 6 x�kj 6 1;

gj1 P 0; gj2 P 0; gj3 P 0;

qj1 P 0; qj2 P 0; qj3 P 0:

Obviously, we can consider different values of wki for each criterion

Yk, and different values of k�.� Global value.

vðxjÞ ¼ Ak x�1j ; . . . ; x�qj

� �¼ x�j 2 ½0;1�;

by solving the following problem

min ð1� kÞDþ kXq

i¼1

wiðgji þ qjiÞ� �

s:t: w1ðgj1 þ qj1Þ 6 D;

. . .

wqðgjq þ qjqÞ 6 D;

x�j þ gj1 � qj1 ¼ x�1j ;

. . .

x�j þ gjq � qjq ¼ x�qj ;

0 6 x�j 6 1

gj1 P 0; . . . ;gjq P 0

qj1 P 0; . . . ;qjq P 0:

3.3. Rating phase

In the exploitation phase, the management team shall classifyand order employees x1, . . ., xn according to:

(1) The collective values for each criterion (see Table 1).(2) The global criteria values (see Table 2).(3) The global values, x�1; . . . ; x�n.(4) The self-values of each employee for each criterion.

The opinions of employees about themselves are taken into ac-count in this stage of the evaluation process due to the fact thatone of the most important aspect of an integral evaluation processis providing performance feedback to employees (see Burns andFlanagan (1955), Latham and Wexley (1981), Banks and Roberson(1985), Bretz et al. (1992), Bernardin et al. (1995), Baron and Kreps(1999), Fletcher (2001) and Fisher et al. (2006), among others). In

Table 2Global criteria values.

Criterion Global criteria values

Y1 x�11 ; . . . ; x�1n

� � � � � �Yq x�q1 ; . . . ; x�qn

1604 R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607

order to improve the employees’ performance, workers need toknow their results in the evaluation process. Most theories of workmotivation indicate that the feedback of the results to employees isan important part of the performance appraisal system becauseemployees must know how well they do their work and how to im-prove their weak points.

Once the company obtains the results of the performance ap-praisal process, the Human Resources Department managementanalyzes them, ranks and sorts employees taking into consider-ation the goals fixed by the company.

4. An illustrative example

In this section we present a simple example in order to showhow companies could carry out their performance process bymeans of the previously proposed model.

With this intention, let us suppose an engineering companydealing in the construction industry is going to carry out a 360-de-gree assessment over their employees.

Before showing the scheme of the performance process for thiscompany, it is remarkable that the market of public works is smallbecause of the big competition among companies in this economysector, as well as the big capital that is necessary to be invested bythe companies in order to develop projects. Due to these facts,these kinds of companies need to efficiently qualify employeesand to plan carefully investments in equipments and tools; to carryout this training, civil engineering companies have turned to theservices of Human Resources consultants to develop and realizeperformance appraisal process.

In this example the company’s goal is to determine the level ofefficiency of each employee in order to develop Human Resourcespolicies and to achieve an effective personnel management. Forthis purpose, the company is carrying out a 360-degree assessmentover their employees of the Civil Engineering Department whichinvolves evaluations from supervisors, collaborators, subordinatesand employees themselves.

To facilitate the process and the calculations we consider, with-out loss of generality, two employees to be evaluated, x1, x2,according to the following criteria (similar to someone used byother companies to carry out performance appraisal (see Fisheret al. (2006))):

� Y1: Knowledge of position. Degree of adaptation betweenemployee and his/her job through the experience, level of edu-cation, specialized training, etc.� Y2: Responsibility. Degree of dedication to the work within

established norms.� Y3: Initiative. Trend to contribute, develop and to carry out new

ideas or methods.� Y4: Communication. Skill of making easier the relationships

among supervisors, collaborators, subordinates, etc.� Y5: Cooperation. Ability to facilitate the teamwork.� Y6: Finishing projects on time. Characteristic of being able to

end punctually a required task work.

The reviewers that are going to take part in the evaluation pro-cess should be people who interact daily with the evaluated

employees. In our example, the Human Resource manager selectsthe following groups of reviewers:

� A set of four supervisors A = {a1, . . . ,a4}.� A set of eight collaborators B = {b1, . . . ,b8}.� A set of twelve subordinates C = {c1, . . . ,c12}.

Members of each group of reviewers express their evaluationsabout workers through the following finite scales of real numbers,according to their knowledge about evaluated employees:

� Supervisors:

S1A ¼ sA1

1 ; . . . ; sA17

� �¼ f1; . . . ;7g; S2

A ¼ sA21 ; . . . ; sA2

7

� �¼ f1; . . . ;7g;

S3A ¼ sA3

1 ; . . . ; sA39

� �¼ f1; . . . ;9g; S4

A ¼ sA41 ; s

A42 ; s

A43

� �¼ f1;2;3g;

S5A ¼ sA5

1 ; . . . ; sA55

� �¼ f1; . . . ;5g and S6

A ¼ sA61 ; . . . ; sA6

5

� �¼ f1; . . . ;5g:

� Collaborators:

S4B ¼ sB4

1 ; . . . ; sB45

� �¼ f1; . . . ;5g; S5

B ¼ sB51 ; . . . ; sB5

9

� �¼ f1; . . . ;9g and S6

B ¼ sB61 ; . . . ; sB6

7

� �¼ f1; . . . ;7g:

� Subordinates:

S4C ¼ sC4

1 ; . . . ; sC45

� �¼ f1; . . . ;5g:

We can note that all collectives of reviewers do not evaluateemployees attending to the same criteria. For example, subordi-nates only evaluate employees with respect to the criterion Y4

due to the fact that they do not have enough knowledge to evalu-ate employees attending to the other criteria.

In Tables 10–12 the assessments provided by the reviewersabout employees x1 and x2 for the different criteria are indicated.

Once all information has been gathered, we follow the schemepresented in Section 2 to obtain the different values which allowthe company to rank employees according to its Human Resourcespolicy.

First, we carry out the normalization phase. All information isstandardized in the interval [0,1] following the method showedin Section 3.1. The following normalized scales are obtained:

� For supervisors:

S1A ¼ �sA1

1 ; . . . ;�sA17

� �¼ 0;

16;26;36;46;56;1

� ;

S2A ¼ �sA1

1 ; . . . ;�sA17

� �¼ 0;

16;26;36;46;56;1

� ;

S3A ¼ �sA3

1 ; . . . ;�sA39

� �¼ 0;

18;28;38;48;58;68;78;1

� ;

S4A ¼ �sA4

1 ;�sA42 ;�s

A43

� �¼ 0;

12;1

� ;

S5A ¼ �sA5

1 ; . . . ;�sA55

� �¼ 0;

14;24;34;1

� ;

Table 3Weight

Y1

Y2

Y3

Y4

Y5

Y6

Table 4Weight

Y1

Y2

Y3

Y4

Y5

Y6

Table 5Collective criteria values for employee x1.

R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607 1605

S6A ¼ �sA6

1 ; . . . ;�sA65

� �¼ 0;

14;24;34;1

� :

Supervisors Collaborators Subordinates

� For collaborators: Y1 v1

Aðx1Þ ¼ 1 – –

Y2 v2Aðx1Þ ¼ 0:33 – –

Y3 v3Aðx1Þ ¼ 0:687 – –

Y4 v4Aðx1Þ ¼ 1 v4

Bðx1Þ ¼ 0:75 v4Cðx1Þ ¼ 0:5

Y5 v5Aðx1Þ ¼ 0:75 v5

Bðx1Þ ¼ 0:687 –

Y6 v6Aðx1Þ ¼ 0:75 v6

Bðx1Þ ¼ 0:833 –

Table 6Collective criteria values for employee x2.

S4B ¼ �sB4

1 ; . . . ;�sB45

� �¼ 0;

14;24;34;1

� ;

S5B ¼ �sB5

1 ; . . . ;�sB59

� �¼ 0;

18;28;38;48;58;68;78;1

� ;

S6B ¼ �sB6

1 ; . . . ;�sB67

� �¼ 0;

16;26;36;46;56;1

� :

Supervisors Collaborators Subordinates

� For subordinates: Y1 v1

Aðx2Þ ¼ 0:5 – –

Y2 v2Aðx2Þ ¼ 0:75 – –

Y3 v3Aðx2Þ ¼ 0:812 – –

S4C ¼ �sC4

1 ; . . . ;�sC45

� �¼ 0;

14;24;34;1

� :

Y4 v4Aðx2Þ ¼ 0:5 v4

Bðx2Þ ¼ 0:5 v4Cðx2Þ ¼ 1

Y5 v5Aðx2Þ ¼ 0:5 v5

Bðx2Þ ¼ 0:625 –

Y6 v6Aðx2Þ ¼ 1 v6

Bðx2Þ ¼ 1 –

Table 7Global criteria values for employees x1 and x2.

x1 x2

v1(x1) = 1 v1(x2) = 0.5v2(x1) = 0.33 v2(x2) = 0.75v3(x1) = 0.687 v3(x2) = 0.812v4(x1) = 0.75 v4(x2) = 0.5v5(x1) = 0.687 v5(x2) = 0.583v6(x1) = 0.75 v6(x2) = 1

After the normalization phase, the aggregation process canstart. The weights in each step of the aggregation phase have beenselected by the company attending to the following principles:

� The importance of the reviewers belonging to the same collec-tive is equal for each criterion (see Table 3).� For computing the global criteria values, the company considers

supervisors and collaborators collectives more important thansubordinates group for all criteria because the subordinatesassessments do not play a fundamental role in the evaluationprocess (see Table 4).� In order to calculate the global value for each employee, the

company assigns each criterion to the same level of significant,wi ¼ 1

6 for each criterion.

We note that different collective criteria values, different globalcriteria values and different global values can be obtained for dif-ferent values of k. Here we only consider k = 0.5 to calculate collec-tive criteria values and global criteria values.

In Tables 5–7, we now show assessments obtained for each em-ployee in the first and second step of the aggregation process, tak-ing into account the different weights, groups of reviewers andcriteria.

s for computing collective global criteria values.

Supervisors fori = 1, . . . ,4

Collaborators fori = 1, . . . ,8

Subordinates fori = 1, . . . ,12

wA1i ¼ 1

4– –

wA2i ¼ 1

4– –

wA3i ¼ 1

4– –

wA4i ¼ 1

4 wB4i ¼ 1

8 wC4i ¼ 1

12

wA5i ¼ 1

4 wB5i ¼ 1

8–

wA6i ¼ 1

4 wB6i ¼ 1

8–

s for computing global criteria values.

Supervisors Collaborators Subordinates

w11 ¼ 1 – –

w21 ¼ 1 – –

w31 ¼ 1 – –

w41 ¼ 2

5 w42 ¼ 2

5 w43 ¼ 1

5

w51 ¼ 1

3 w52 ¼ 2

3–

w61 ¼ 1

2 w62 ¼ 1

2–

In this way, we obtain a collective criteria value for each em-ployee and each criterion, and a global criteria value for each em-ployee and each criterion.

In order to show in detail the last step of the aggregation pro-cess, we now present the procedures of computing the global valuefor each employee.

� Employee x1:

vðx1Þ ¼ Ak x�11 ; x�21 ; x

�31 ; x

�41 ; x

�51 ; x

�61

� �¼ x�1 2 ½0;1�;

by solving the following problem

min ð1� kÞDþ kX6

i¼1

wiðg1i þ q1iÞð Þ

s:t: w1ðg11 þ q11Þ 6 D;

w2ðg12 þ q12Þ 6 D;

w3ðg13 þ q13Þ 6 D;

w4ðg14 þ q14Þ 6 D;

w5ðg15 þ q15Þ 6 D;

w6ðg16 þ q16Þ 6 D;

x�1 þ g11 � q11 ¼ x�11 ;

x�1 þ g12 � q12 ¼ x�21 ;

x�1 þ g13 � q13 ¼ x�31 ;

x�1 þ g14 � q14 ¼ x�41 ;

x�1 þ g15 � q15 ¼ x�51 ;

x�1 þ g16 � q16 ¼ x�61 ;

0 6 x�1 6 1;g11 P 0; g12 P 0; g13 P 0; g14 P 0; g15 P 0; g16 P 0;q11 P 0; q12 P 0; q13 P 0; q14 P 0; q15 P 0; q16 P 0:

� Employee x2:

vðx2Þ ¼ Ak x�12 ; x�22 ; x

�32 ; x

�42 ; x

�52 ; x

�62

� �¼ x�2 2 ½0;1�;

Table 12Customers assessments for Y4 and each employee.

c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11 c12

x1 sC45 sC4

2 sC44 sC4

1 sC43 sC4

5 sC42 sC4

5 sC45 sC4

1 sC44 sC4

3

x2 sC45 sC4

5 sC45 sC4

5 sC45 sC4

4 sC44 sC4

2 sC44 sC4

4 sC44 sC4

5

1606 R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607

by solving the following problem

min ð1� kÞDþ kX6

i¼1

wiðg2i þq2iÞð Þ

s:t: w1ðg21þq21Þ6 D;w2ðg22þq22Þ6 D;

w3ðg23þq23Þ6 D;

w4ðg24þq24Þ6 D;

w5ðg25þq25Þ6 D;

w6ðg26þq26Þ6 D;

x�2þg21�q21 ¼ x�12 ;

x�2þg22�q22 ¼ x�22 ;

x�2þg23�q23 ¼ x�32 ;

x�2þg24�q24 ¼ x�42 ;

x�2þg25�q25 ¼ x�52 ;

x�2þg26�q26 ¼ x�62 ;

06 x�2 6 1;g21 P 0; g22 P 0; g23 P 0; g24 P 0; g25 P 0; g26 P 0;q21 P 0; q22 P 0; q23 P 0; q24 P 0; q25 P 0; q26 P 0:

In order to show how k can be used by companies to decideabout the degree of efficiency and equity in the performance ap-praisal process, we consider different values of k to calculate theglobal values for employees x1 and x2 (see Tables 8 and 9).

We can note that:

Table 8Global values for employee x1.

k = 0 k = 0.5 k = 1

Global values v(x1) = 0.665 v(x1) = 0.687 v(x1) = 0.687

Table 9Global values for employee x2.

k = 0 k = 0.5 k = 1

Global values v(x2) = 0.75 v(x2) = 0.75 v(x2) = 0.583

Table 10Supervisors assessments for each employee and each criterion.

x1 x2

Y1 Y2 Y3 Y4 Y5 Y6 Y1 Y2 Y3 Y4 Y5 Y6

a1 sA17 sA2

4 sA39 sA4

3 sA55 sA6

5 sA13 sA2

4 sA39 sA4

3 sA53 sA6

5

a2 sA16 sA2

3 sA37 sA4

3 sA55 sA6

4 sA12 sA2

7 sA38 sA4

2 sA53 sA6

5

a3 sA17 sA2

2 sA36 sA4

3 sA53 sA6

4 sA12 sA2

6 sA36 sA4

2 sA54 sA6

5

a4 sA17 sA2

3 sA34 sA4

2 sA53 sA6

4 sA11 sA2

4 sA36 sA4

1 sA53 sA6

4

Table 11Collaborators assessments for each employee and each criterion.

x1 x2

Y4 Y5 Y6 Y4 Y5 Y6

b1 sB44 sB5

9 sB66 sB4

4 sB58 sB6

5

b2 sB44 sB5

8 sB67 sB4

4 sB56 sB6

5

b3 sB45 sB5

7 sB65 sB4

4 sB57 sB6

5

b4 sB44 sB5

5 sB66 sB4

4 sB55 sB6

5

b5 sB45 sB5

6 sB64 sB4

4 sB57 sB6

5

b6 sB43 sB5

7 sB65 sB4

4 sB54 sB6

4

b7 sB43 sB5

4 sB66 sB4

5 sB54 sB6

5

b8 sB44 sB5

5 sB66 sB4

3 sB54 sB6

4

� If companies would like to obtain an efficiency ranking proce-dure, they will consider k = 0. In this case,

vðx1Þ ¼ 0:665 < 0:75 ¼ vðx2Þ:

� If companies would like to obtain an equitable ranking proce-dure, they will consider k = 1. In this case,

vðx1Þ ¼ 0:687 > 0:583 ¼ vðx2Þ;

and the order of the ranking changes.

5. Conclusions

Due to the fact that the 360-degree performance appraisal pro-cess incorporates assessments from different groups of reviewers,it would be necessary to use a formal methodology to suitablymanage such all information and to properly carry it out. Moreoverand taking into account that in traditional 360-degree performanceappraisal methods, opinions are expressed in the same expressionscale without considering the different knowledge levels of thereviewers on employees, in this paper we propose a flexible 360-degree performance appraisal model which provides the followingadvantages:

� It can be adapted to different companies conditions in order tofacilitate the Human Resources decision making processes.� It is based on the Decision Making Theory.� It has been developed on a flexible framework capable of work-

ing with different evaluation scales. Specifically, the collectivesof reviewers could use different numerical scales to evaluateemployees attending to each criterion in order to improve theprecision of the process results and taking into account thedegree of knowledge that the different appraisers have aboutemployees.

Since our proposal of the 360-degree appraisal process takesinto account the assessments of different collective of reviewers,a variety of scenarios based on social principles could be consid-ered; for instance, the government of the majority or the consider-ation to minorities. To deal with these kinds of scenarios, we haveproposed a distance-based framework (Lp-metrics) for aggregatingindividual valuations. In this context, the p parameter has a prefer-ential interpretation in terms of the sensitiveness towards assess-ments’ dispersion.

The computational complexity of a general Lp-metric basedmodel can be avoided by an Extended GP formulation. Now, thecontrol parameter, k, can be interpreted as a marginal rate of sub-stitution between the ‘‘principle of majority” and the ‘‘principle ofminority”. Furthermore, the required computational burden is verylow. In fact, we only need to solve Linear Programming problems.Thus, different values of w and k can be used for reaching aggrega-tion values by means of different points of view and sensitivenesstowards assessments’ dispersion. A computational experiment formaking sensitivity analysis on these parameters could be consid-ered as future research.

We have obtained not only a global assessment for each em-ployee, but intermediate values according to the opinions of eachset of reviewers and criteria, and a global assessment for each cri-terion. It is worth emphasizing that the proposed model is quite

R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607 1607

flexible in allowing the management team to know how to aggre-gate the individual opinions.

Acknowledgements

The authors thank the two anonymous referees for theircomments and suggestions in helping us to improve the paper.de Andrés and García-Lapresta gratefully acknowledge the fundingsupport of the Spanish Ministerio de Ciencia e Innovación (ProjectECO2009–07332), ERDF and the Junta de Castilla y León (Conse-jería de Educación; Project VA092A08).

References

Antes, J., Campen, L., Derige, U., Titza, C., Wolle, G.D., 1998. A model-based decisionsupport system for the evaluation of flight schedules for cargo airlines. DecisionSupport Systems 22, 307–323.

Arfi, B., 2005. Fuzzy decision making in politics: A linguistic fuzzy-set approach.Political Analysis 13, 23–56.

Banks, C.G., Roberson, L., 1985. Performance appraisers as test developers. Academyof Management Review 10, 128–142.

Baron, J.N., Kreps, D.M., 1999. Strategic Human Resources, Frameworks for GeneralManagers. Wiley & Sons, New York.

Bernardin, H.J., Kane, J.S., Ross, S., Spina, J.D., Johnson, D.L., 1995. Handbook ofHuman Resources Management. Blackwell, Cambridge.

Bouyssou, D., Marchant, T., Pirlot, M., Perny, P., Tsoukiàs, A., 2000. Evaluation andDecision Models: A Critical Perspective. Kluwer Academic Publishers.

Bretz, R.D., Milkovich, G.T., Read, W., 1992. The current state of performanceappraisal research and practice: Concerns, directions and implications. Journalof Management 18, 321–352.

Burns, R.K., Flanagan, J.C., 1955. The employee performance record: A new appraisaland development tool. Harvard Business Review 5, 95–102.

Clemen, R.T., 1995. Making Hard Decisions. An Introduction to Decision Analysis.Duxbury Press.

Cleveland, J.N., Murphy, K.R., Williams, R., 1989. Multiple uses of performanceappraisal: Prevalence and correlates. Journal of Applied Psychology 74, 130–135.

de Andrés, R., García-Lapresta, J.L., Martínez, L., 2010. Multi-granular linguisticperformance appraisal model. Soft Computing 14, 21–34.

Dopazo, E., González-Pachón, J., 2003. Computational distance-basedapproximation to a pairwise comparison matrix. Kybernetika 39, 561–568.

Edwards, M., Ewen, E., 1996. Automating 360 degree feedback. HR focus 73, 3.Fisher, C., Schoenfeldt, L.F., Shaw, J.B., 2006. Human Resources Management.

Houghton Mifflin Company, Boston.Fletcher, C., 2001. Performance appraisal and management: The developing

research agenda. Journal of Occupational and Organization Psychology 74,473–487.

Frunzi, G., Savini, P.E., 1996. Supervision: The Art of management. Prentice Hall,New Jersey.

González-Pachón, J., Romero, C., 1999. Distance-based consensus methods: A goalprogramming approach. OMEGA International Journal of Management Science27, 341–347.

González-Pachón, J., Romero, C., 2001. Aggregation of partial ordinal rankings: Aninterval goal programming approach. Computers and Operation Research 28,827–834.

González-Pachón, J., Romero, C., 2006. An analytical framework for aggregatingmulti-attribute utility functions. Journal of Operational Research Society 57,1241–1247.

González-Pachón, J., Romero, C., 2007. Inferring consensus weights from pairwisecomparison matrices without suitable properties. Annals of OperationsResearch 154, 123–132.

Ignizio, J.P., Caliver, T., 1994. Linear Programming. Prentice-Hall, Englewood Cliffs,NJ.

Jarman, M., 1999. Complete turnaround 360-degree evaluations gaining favour withworkers management. Arizona Republic, D1.

Kerr, J.L., 1985. Diversification strategies and managerial rewards: An empiricalstudy. Academy of Management Journal 28, 155–179.

Latham, G.P., Wexley, K., 1981. Increasing Productivity through PerformanceAppraisal. Addison-Wesley.

Likert, R., 1932. A technique for the measurement of attitudes. Archives ofPsychology 140, 1–55.

Martínez, L., 2007. Sensory evaluation based on linguistic decision analysis.International Journal of Approximated Reasoning 44, 148–164.

Miner, J., 1998. Development and application of the rated ranking technique inperformance appraisal. Journal of Occupational Psychology 6, 291–305.

Pfau, B., Kay, I., 2002. Does 360-degree feedback negatively affect companyperformance? HR Magazine 47, 56–60.

Romero, C., 1991. Handbook of Critical Issues in Goal Programming. PergamonPress, Oxford.

Romero, C., 2001. Extended lexicographic goal programming: A unifying approach.OMEGA, The International Journal of Management Science 29, 63–71.

Simon, H., 1957. Models of Man. John Wiley and Sons, New York.Yu, P., 1973. A class of solutions for group decision problems. Management Science

19, 936–946.