THE "BIG FIVE" PERSONALITY FACTORS IN THE IPI AND MMPI: PREDICTORS OF POLICE PERFORMANCE

THE "BIG FIVE" PERSONALITY FACTORS IN THE IPI AND MMPI: PREDICTORS OF POLICE PERFORMANCE

JOSE M. COR"A, MARY L DOHERTY, NEAL SCHMI?T Michigan State University

GARY KAUFMAN, RICHARD G. SMITH Behavioral Science Section

Michigan State Police

Rational and empirical linkages were formed between the "Big Five" personality factors and two personality inventories: the Inwald Per- sonality Inventory (IPI), which is a personality inventory designed especially for use in the selection of corrections officers, and the Min- nesota Multiphasic Personality Inventory (MMPI). A criterion-related validation of the study was then conducted to assess the validity of these two measures of the Big Five in predicting various measures of police performance. Results indicated that while both inventories provided adequate measures of Neuroticism, Extraversion, Agreeableness, and Conscientiousness, neither inventory consistently contributed incremental validity over the Civil Service exam.

Psychological testing has been used in law enforcement for many years in an attempt to screen out those candidates who would be ill- suited to law enforcement jobs. The ability to screen out these candidates presumably offers law enforcement agencies two types of benefits (Inwald, 1988; Murphy, 1972). First, it saves these agencies the time and money required to train new recruits who subsequently fail during either training or probationary periods. Second, it is felt that psychological testing can identify those persons who are likely to misuse force or weapons on the job (Inwald & Shusman, 1984).

MMPI Use

Of tests that have been used for these purposes, the most common has been the Minnesota Multiphasic Personality Inventory (MMPI). Some studies have indicated that the responses of candidates for police positions exhibit identifiable profiles on the MMPI. For example, police officer candidates typically show defensiveness on the validity scales

Correspondence and requests for reprints should be addressed to Jose M. Cortina, Department of Psychology, Michigan State University, East Lansing, MI 48824-1117.

COPYRIGHT @ 1992 PERSONNEL PSYCHOuxiy, INC.

119

120 PERSONNEL PSYCHOLOGY

(McAllister, 1977; Sacuzzo, Higgins, & Lewandowski, 1974) and ele- vations on the Psychopathic Deviate (Pd) scale. However, in his review of the literature, Butcher (1979) reported that most efforts to link MMPI scores to job performance have been unsuccessful (Colarelli & Siegel, 1964, Gottesman, 1969; Hooke & Krauss, 1971; McAllister, 1977; Nowicki, 1966). One exception was a 20-year follow-up study by k e n , Snibbe, and Montgomery (1973) in which the researchers found that scores on the Hypomania (Ma) scale were related to police officers’ pre- ventable auto accidents.

In the last 10 years, there have been growing concerns over the use of the MMPI in the selection of law enforcement officers. These concerns seem to stem from the fact that the MMPI was not designed specifically for selecting police officers and the possibility that this might account for the generally low correlations that have been found between the MMPI and police performance (Mass, 1979). It was in response to these concerns that the Inwald Personality Inventory (IPI) was developed (Inwald, Knatz, & Shusman, 1983).

Development and Use of the IPI

The IPI is a 310 item true-false questionnaire that was developed specifically for use by law enforcement agencies in selecting new officers. The test attempts to assess the psychological and emotional fitness of recruits as well as some of their job-relevant behavioral characteristics (Inwald et al., 1983). The development of the IPI was predicated on the notion that items written so as to be relevant to police occupations and personnel would be better predictors of police job performance than items from other more general inventories such as the MMPI, which was developed to diagnose a variety of psychological disorders and includes many items that may appear bizarre when asked in a selection situation. Schneider and Schmitt (1986) have suggested that the typical failure of personality tests to improve personnel selection may be due to the fact that most tests of personality-criterion relationships have not evaluated personality measures that were based on job analyses or even a consid- eration of the occupation(s) with which the test might be used. The IPI consists of 26 scales that include six specific external behavior scales, five attitude and temperament scales, eight internal conflict scales, six inter- personal conflict scales, and one validity scale.

Attempts to validate the IPI for use as a selection instrument have generally yielded favorable results. In a concurrent validity study, In- wald et al. (1983) found that the IPI accurately predicted 72% of the male officers and 83% of the female officers who were terminated from the department as opposed to 62% and 66% respectively for the MMPI.

JOSE M. COKITNA ET AL. 121

The IPI was also found to be superior in predicting minority turnover with no evidence of differential validity. The same study also found the IPI to be superior in predicting more specific job-relevant criteria, such as absences, lateness, and derelictions (disciplinary interviews administered by a supervisor). The IPI manual reports separate male and female norms, but there are only small differences in means and standard deviations across gender subgroups.

In a predictive validity study, Inwald (1988) found separate discrim- inant function equations for the IPI and the MMPI that predicted 68% of the officers who were subsequently terminated. An equation formed from the two inventories combined was found to predict 21 of the 28 terminated officers, or 75%. The author concluded that psychological screening tests, if used properly (i.e., with empirically derived prediction equations), are quite useful in predicting law enforcement officer turnover. The Inwald (1988) study resulted in the development of a risk score,l which was the weighted combination of several IPI scales. This risk score was recommended as the basis of screening applicants for law enforcement jobs. Because of its purely statistical nature and the fact that it was validated only for turnover, however, the risk score does little to enhance our understanding of the personality constructs involved in police performance.

While the results of studies on the predictive validity of the IPI have been encouraging, generally disappointing results concerning the validity of personality scales have been reported for other personality measures across various occupations (Schmitt, Gooding, Noe, & Kirsch, 1984). In addition to the fact that personality tests have not been developed with personnel selection applications in mind, a second possible reason for these discouraging results is that personality researchers have lacked any well accepted taxonomy for the classification of personality measures. Without some unifying framework, it has been difficult to ascertain whether there are consistent and meaningful personality- criterion relationships.

Development of a Personality Tmnomy

In the last decade personality psychologists have reached some con- census with respect to the structure of personality (Digman, 1990). Five

’ ‘Included among the ‘‘Extra Scales’’ presented in lhble 1 is the IPI Risk variable. This is a weighted composite of the various IPI subscales and, therefore, could not be assigned to any of the “Big Five.”


factors (Extraversion, Agreeableness, Conscientiousness, Emotional Sta- bility, and Openness to Experience) appear robust across many theoret- ical frameworks using different instruments in a variety of samples. Fur- ther, these five factors appear to be relatively uncorrelated with measures of Cognitive ability (McCrae & Costa, 1985).

Recently, Barrick and Mount (1991) used this five-factor model as the basis to reorganize and reanalyze data on the criterion-related validity of personality tests. Conscientiousness showed modest but consistent relationships with all job performance criteria. The average correlation between criteria and Conscientiousness corrected for unreliability in the predictor and the criterion and for range restriction was .22. Results for Openness to Experience and Extraversion were similarly positive with respect to the prediction of training proficiency criteria, and measures of the other personality dimensions appear to be valid predictors of some criteria for some occupations. Measures of police performance were correlated significantly (in the sense that 90% credibility values did not include zero) with all five personality factors except the Openness to Ex- perience factor. Average corrected correlations were .22 for Conscien- tiousness and .09 or .10 for the other three significant factors. While not large, these validities can be practically useful especially if the personality measures are uncorrelated with measures of other constructs, such as cognitive ability, that are more frequently used.

The present study represents an attempt to employ the five-factor model of personality to understand better the relationship between IPI and MMPI scales and various criteria of police performance. Based on the Bamck and Mount (1991) meta-analyses, we expect scales that measure the Conscientiousness factor will be the best predictors of the various performance measures. Extraversion, Emotional Stability, and Agreeableness should also be correlated with performance but these validities should be lower. Second, we compare the validities of the IPI and MMPI.

As indicated above, the IPI was constructed for, and has been used, primarily with police or protective occupations. Given the earlier disappointing results for the MMPI (Butcher, 1979) and the relatively favorable results for the IPI (Inwald, 1988; Inwald et al., 1983; Inwald & Shusrnan, 1984), it is expected that the IPI will add significantly to the predictability of performance above and beyond the predictability af- forded by MMPI measures. The reverse, however, should not occur; that is, the MMPI will not add to validity above that of the IPI scales. We include estimates of the validity of the personality measures across a variety of criteria including peer and supervisory evaluations during a 6-month training period, academic training performance, turnover, and job performance ratings. Assessing both training success and on-the-job

JOSE M. CORTINA ET AL. 123

performance allows for an evaluation of both goals associated with the introduction of personality testing for police officer selection mentioned above.

Previous published studies on the IPI have often focussed on the prediction of turnover (e.g., Inwald, 1988) though some previous research also assessed the validity of the IPI relative to other criteria (e.g., Inwald & Shusman, 1984). All analyses assess the incremental validity of the personality tests relative to the validity of a Civil Service exam, proba- bly best characterized as a measure of cognitive ability, and a structured interview. Finally, the possibility of differential prediction for minority and majority subgroups was evaluated.

Method

Sample

The sample consisted of 314 state police recruits from four different training schools who entered a midwestern state police academy between October, 1986 and September, 1988. This recruit sample consisted of 228 (72.6%) males and 86 (27.4%) females with minorities comprising just under 31% of the sample (n = 97).

Predictors

Recruits were required to complete both the MMPI and the IPI the first evening of the state police training academy. Their responses on these tests were used to predict subsequent performance but were not used in any way to select personnel during the course of the validation effort. Recruits had been screened by both a Civil Service examination and a panel interview.

MMPI and IPI

In order to use the five-factor taxonomy described above, it was nec- essary to make judgments about which of these five personality factors, if any, was being measured by each of the MMPI and IPI scales. There have been few previous attempts to rationally link the MMPI to the “Big Five.” Costa, Zonderman, McCrae, and Williams (1985) compared the content of MMPI items to the descriptions of the “Big Five” presented by Norman (1963) and concluded that the MMPI provided measures of four of the factors with Conscientiousness excluded. No information was provided on the agreement criteria that led to this conclusion. Johnson,


Butcher, Null, and Johnson (1984) reached similar conclusions, however, it was once again difficult to glean from the study the agreement criteria that were used to reach their conclusions. Costa, Busch, Zon- derman, and McCrae (1986) correlated MMPI factor scales with measures of the “Big Five” and concluded that Neuroticism, Extraversion, Openness to l3perience, and Agreeableness were well represented in the items of the MMPI. No previous research in the area of personality has attempted to link the IPI to the “Big Five.”

Each of the five authors of the present paper used a description of the five factors taken from Digman (1990) and brief descriptions of each of the scales in the two personality inventories (the authors were also familiar with the item content of the scales) to make judgments about which of the five factors each scale indexed. If three of the five judges agreed with respect to the factor being measured, it was included as one of the scales measuring that component. The choice of 60% agreement as the cutoff was somewhat arbitrary since little information on how agreement has been established in similar studies (e.g., Costa et al., 1985; Johnson et al., 1984) is available. Nevertheless, 60% agreement among judges on which of six categories (the “Big Five” and a Miscellaneous category) a scale should be assigned is substantially better than chance agreement and reflects moderate consistency across judges. Given the exploratory nature of these classifications, especially with regards to the IPI, 60% agreement was judged to be adequate. A list of the scales and the factor to which they were assigned is provided in lhble 1. Further evaluation of the empirical meaningfulness of our categorization is represented by the internal consistencies of our five factors and the intercorrelations of these factors (see lhble 2 below). The list in lhble 1 also includes a list of scales about which we either disagreed or about which we agreed that the scale did not define any of the five factors. The validity of these “extra” scales was also evaluated in the analyses described below. Raw scores on the personality scales defining a particular factor were summed to form factor scores.2 These five factor scores for the two inventories were the primary predictors of interest.

Civil Service Eramination

The written Civil Service examination (CSE) is a knowledge-based multiple choice exam perhaps best characterized as a test of cognitive

2Raw scores were used to compute “Big Five” factor scores for two reasons. First, it was suggested to us by someone familiar with the previous work done on the IPI that raw scores would be more appropriate given the nature of the normative data done on the IPI. Second, correlations between factor scores computed from raw scores and factor scores computed from standardized scores were all greater than .95.

JOSE M. COKITNA ET AL. 125

ability. It consists of three sections (a) a reading comprehension section in which examinees were asked to read a paragraph and then answer questions about that paragraph, (b) an accuracy of observation section in which examinees were presented with a scene and then asked a se- ries of questions about what they had observed, and (c) a writing exam. All items are in true-false or multiple choice format. In addition, all recruits were evaluated in an oral board interview conducted by three interviewers who made independent judgments about the qualifications of the candidate. Specifically, candidates were rated on seven dimensions such as integrity, motivation, and communication skills. We were primarily interested in the incremental validity of the personality measures over other commonly used selection techniques such as measures of cognitive ability and structured interviews, hence the composite scores on the CSE (panel interview plus written exam) were entered first in the hierarchical regression analyses described below.

Criterion Measures

The different types of performance data used in this study were final training ratings, peer ratings, individual grade point average (GPA), probationary ratings, counseling card information, and turnover.

Gmdepoint averuge (GPA). Individual GPA is a measure of individual performance in training courses at the academy. The recruits are required to take weekly exams in a number of courses such as first aid, firearms, and criminal investigation. A composite of scores on these exams are used to compute their GPA. Because GPA data were available only in composite form, no reliability analyses were possible.

Total turnover. If an individual left the academy either during or after training for any reason, that person was assigned a score of one for this measure; otherwise, they were given scores of zero.

Probation ratings. Once a month during the recruits’ 6-month on-the- job probation period, supervisors rated the recruits on 14 dimensions (e.g., punctuality, professionalism, fairness, etc.). Ratings were made on 4-point scales with a high score indicating satisfaction with the rated behavior and a low score indicating dissatisfaction. Dimensions rated as “not observed” were coded as missing data. These ratings are part of the normal administrative workload of the supervisors and have been routine practice in the state police units for several years. Dimensions were behaviorally defined, but anchors ranged from “Satisfactory” to “Unsatisfactory.” No special rater training was provided for this project.

The 14 dimensions were highly correlated (coefficient alphas by month ranged from .86 to .91), so all 14 ratings were combined within each month. Further, these composite ratings were highly stable across


months 1 to 6 (alpha for the 6 months combined was .92), so the monthly ratings were further combined to form a single index of rated on-the-job performance during each police officer’s first 6 months at work. No index of interrater reliability or agreement could be obtained.

Peer evaluations. Recruits were asked to provide ratings of their peers’ performance on 10 dimensions twice during their training at the academy. These dimensions were similar to those used to evaluate their job performance during their on-the-job probationary period. The ratings were performed on 9-point, “low” to “high” scales. Ratings from 12 peers were available for most recruits although some recruits had been rated by as many as 19 peers. The peer evaluations from one of the schools were missing so peer evaluation data were available only on 149 of the sample. An estimate of interrater reliability was computed for each set of ratings. The interrater reliability estimate (coefficient alpha) for the first set of ratings (Time 1) was .81 and the estimate for the second set of scores (Time 2) was 8 5 . Composites of these ratings computed at each of the two time points correlated .65, so Time 1 and Time 2 ratings were combined to form a single peer evaluation composite. Coefficient alpha for this composite was .98.

Final mining mtings. Supervisors were asked to rate the recruits as satisfactory (1) or unsatisfactory (0) on 21 dimensions at the end of their dmonths training at the academy. The ratings were combined into one scale, and the internal consistency reliability of this scale was .74. No measure of interrater reliability was available.

Counseling c a d . During the training period, recruits were coun- selled by some member of the academy staff whenever they had inter- personal or personal difficulties or when they had academic problems. As part of their report on these counselling sessions, the academy st& were required to rate the recruit on 12 dimensions such as integrity, motivation, maturity, and leadership abilities. Responses were made on a 4-point scale ranging from “unsatisfactory” to “excellent.” Responses of “not observed” were coded as missing data. Interrater reliability was es- timated by analyzing counseling card information from a police school not used subsequently in this study. The resulting estimate of interrater reliability (based on two raters) was .45.

Most of the recruits from the schools included in this study had ratings on at least nine counseling cards. Internal consistency reliability analyses were computed for the ratings on each of the nine counseling cards, rehulting in reliability coefficients ranging from .92 to .96. Inter- correlations of the means for each counseling card (which always differed with respect to the time at which the card was completed and occasion- ally also differed with respect to which member of the training academy


staff provided the rating) suggested that cards 1-9 could be combined into one measure (alpha = .93)

Data Analyses

The five factor composites for both MMPI and IPI measures were computed and intercorrelated. Wherever possible, internal consistency estimates were calculated. Correlations among the various criterion ele- ments and correlations between the criteria and the predictors were also calculated. The latter represent the validity coefficients of the IPI and MMPI measures of the five personality constructs and the Civil Service exam.

To test the validity of the entire set of predictors as well as the incremental validity of both inventories over and above the Civil Service exam and the other inventory, we conducted hierarchical least squares regression analyses for each dependent variable except turnover. To test the incremental validity of the MMPI factors, we conducted a hierarchical regression in which variables were entered as follows 1) Civil Service exam score, 2) the five scores from the IPI, and 3) the five scores from the MMPI. In testing the incremental validity of the IPI, the order of entry of the second and third blocks of variables was reversed. Similar analyses were also conducted in which the five scores from both inventories were entered together in one step. In addition, for each dependent variable, we performed a regression analysis in which we entered on a final step all those scales not judged to be measures of one of the five personality constructs. Because of the dichotomous nature of the turnover measure, we conducted stepwise hierarchical logistic regressions entering variables in the same order as described above.

Finally, we conducted hierarchical moderated regression analyses (Bartlett, Bobko, Mosier, & Hannan, 1978; a h e n & Cohen, 1983) to ascertain whether there was any evidence for differential prediction between minority and majority groups.

Results

Scale Assignment and Development

The results of the assignment of MMPI and IPI scales to the “Big Five” factors are presented in Tmble 1. As stated above, at least three of the five authors had to agree that a particular scale was a measure of one of the five factors.

One point that should be made about this table is that, while some of these scale assignments are immediately obvious, such as the judgment


TABLE 1 ‘Big Five” P e r S o M ~ Factors and the Scales from the MMPI

and IPI That Werehigned to Them

”Big Five” factors Scales

Neuroticism IPI Hyperactivity, Illness Concerns, ’Reatment Programs, Anxiety, Phobic Personality, Obsessive Personality, Depression

MMPI Hypochondria&, Depression, Psychasthenia

IPI Loner ljp, Lack of Assertiveness MMPI Social Introversion IPI Rigid ljp MMPI Masculine/Fcrninine

IPI ’Rouble with the Law, Antisocial Attitudes, Inter-

Extraversion

O p e ~ e a ~ to experience

Agrccableness personal Difficulties, Undue Suspiciousness, Sexual Concerns, Spouse/Mate Conflicts

MMPI Paranoia IPI Alcohol Use, Drug Use, Driving Violations, Job

DifEdties, Absence Abuse, Substance Abuse MMPI Psychopathic Deviate, MacAndrews Alcoholism

IPI Family Conflicts, Unusual Experienced Thoughts,

MMPI GValidity, K-Validity, F-Validity, Hysteria,

Conscientiousness

Extra scales Risk, ’I)pe A Personality

Hypomania, Schizophrenia

that anxiety and depression are measures of Neuroticism, others are not so obvious. For example, the MMPI Psychopathic Deviate scale was assigned to the Conscientiousness factor. This assignment makes sense in light of the description of people who m r e high on the Psychopathic Deviate scale offered by The MMPI Notebook (1966), which is that they show “. . . a disregard for social values, an inability to profit from experience, and a difficulty in maintaining personal relationships” (p. 9). The authors felt that this assignment was appropriate because a person who scores high on the MMPI Psychopathic Deviate scale is described as having little or no conscience and is, therefore, lacking in conscientiousness. Similar judgments were made in the case of several other scales in both inventories.

A second point to be made about ’Ihble 1 is that some of the “Big Five” factors are better represented by the two inventories than others. Neuroticism, Agreeableness, and Conscientiousness are measured by 10, 7, and 8 scales respectively while Extraversion and Openness to Experience are measured by 3 and 2 scales respectively. Extraversion, however, was an explicit focus of the developers of both inventories. This is clearly evidenced by the titles of the three scales that were assigned to

TABLE

2 M

eans

, Sta

ndar

d D

evia

tions

, and

Inte

rcom

ktio

ns Am

ong

“Big

Five

” Fac

tors

as M

easm

d by

the I

PI a

nd MMPI

Scal

es

1. IP

I N

euro

ticism

2.

IPI

Extr

aver

sion

3. IP

I O

penn

ess

4. IP

I A

gree

able

ness

5.

IPI

Con

scie

ntio

us

6. MMPI N

euro

ticism

7.

MM

PI E

xtra

vers

ion

8. M

MPI

Ope

nnes

s 9.

MM

PI A

gree

able

ness

10

. MM

PI C

onsc

ient

ious

11

. Civ

il Ser

vice

exam

1

Mea

n SD

31.4

16

.0 .8

fP

7.8

3.6

.40

6.9

2.3

.48

19.2

11.1

.8

4 16

.3

7.6

.67

30.5

11

.8

.77

23.2

8.0

.49

26.8

5.5

.26

9.6

2.6

.23

38.1

5.

5 .40

94.1

5.

0 -.3

2

2 3

.49

.13 .30

.13

.53

.72

.07

.08 .01

-.12

.62

.29

24

.3

2 .0

1 - .0

5 .2

1 -.1

8

lo u E) m ic

4 5

6 7

8 9

.80

.68

.70

.61

.49

.68

.43

.16

.65

.12

.13

.25

.06

.42

.42

.32

.05

.05

.31

.28

F 1 T .1

3 .2

0 .3

6 .0

3 .1

2 3

-28

-.23

-.35

-.27

-.06

-.38

-20

Not

e: N =

307

for C

Ml S

ervi

ce e

xam

, 311

othe

rwise

; cor

rela

tions

> .1

2 ar

e sig

nific

ant, p

<.0

5.

aFor

thos

evar

iabl

es fo

r whi

ch it

was

pos

sible

to co

mpu

te (i

.e., f

or th

ose v

aria

bles

whi

ch w

ere c

ompu

ted f

rom

mul

tiple

MM

PI or

IPI s

cale

s), c

oefic

ient

al

phas

are p

rese

nted

on

the

diag

onal

.


this factor: IPI Loner T j p , IPI Lack of Assertiveness, and MMPI Social Introversion. It would appear, therefore, that Openness to Experience is the only one of the “Big Five” factors that is not well represented by either of the inventories; only a single scale in each of the inventories was judged to be a measure of Openness to Experience.

’Igble 2 shows the means, standard deviations, and intercorrelations for the 11 predictors (5 IPI, 5 MMPI, and the Civil Service exam). Sev- eral observations about these data are important. First, the two scales judged to be measures of Openness to Experience are unrelated. This lack of convergent validity suggests that Openness to Experience may not be represented in either the IPI or the MMPI, or in one but not the other. The correlation between measures of Agreeableness is also very low; only the measures of Neuroticism and Extraversion show good cross-inventory correlations.

A second observation concerning the correlations in ’Igble 2 is that four of the five factors are at least moderately correlated. Only the measures of Openness to Experience are relatively uncorrelated with other factors. The Neuroticism and Agreeableness measures in particular are highly correlated with each other and with the three other factors. That this is true for the MMPI is perhaps not surprising since many of the MMPI scales contain “bizarre” item content often associated with Neu- roticism and “anti-social behavior” item content often associated with low levels of Agreeableness. However, correlations between the “Big Five” factors as defined by the IPI were also substantial; they were, in fact, higher than similar correlations between MMPI factors.

A final point to be made about Table 2 has to do with the correlations between the various personality measures and the Civil Service exam. Several of these correlations are statistically significant and relatively large (>.30). Since we were interested in the incremental validity of personality measures above the more frequently used cognitive ability measures and structured interviews, all subsequent regression analyses were performed with the Civil Service exam partialled out of the equation.

Table 3 presents the means, standard deviations, and intercorrelations among the six criteria. ’Ikro points should be made about this table. First, Final ltaining Ratings show very little variance. Second, each of the six criteria is correlated significantly (p <.01) with each of the other criteria. These correlations, while statistically significant, are rather low. This finding, however, might be expected based on other research on criterion dimensionality (e.g., Seashore, Indik, & Georgopoulos, 1960; Harris & Schaubroeck, 1988). One would also expect relatively low correlations based on the nature of the criteria. Probation ratings, for example, are actual on-the-job performance ratings, counseling cards are

JOSE M. COKI’INA ET AL. 131

TABLE 3 Means, Standard khtions, and Intercorrelations Among the Six Criteria

Variable N Mean SD PR PE CC “RT GPA TRN

Robationratings(PR) 230 3.6 .33 Peerevaluation(PE) 149 8.0 .46 .34 Counselingcards(CC) 241 2.3 .35 .19 5 2 ’Rainingratings(TI0 241 2.0 .06 .29 .59 .31 Gradepointavg. (GPA) 241 89.0 4.24 .40 .32 .37 .41 Hunoveer (TRN) 314 .27 .44 -.42 -.36 -.19 -.18 -.27

No&: All of the above correlations are significant, pc.01.

only completed when a trainee is having some difficulty with the training process or situation, h a l training ratings are completed by supervisors at the end of training, and peer evaluations are, of course, completed by peers.

’Ttible 4 presents the correlations between the 10 personality-based predictors and the six criteria. The majority of these correlations are negative, as expected, since high scores on most personality dimension indicated a “negative” aspect of the respondents personality. For example, if a person scored high on the IPI conscientiousness factor (ie., the person was not very conscientious), then that person scored high on the scales measuring conscientiousness such as absence abuse and drug use. It seems reasonable to expect such a person to have low supervisor and peer ratings. Also of note in this table is the fact that MMPI Neuroticism, MMPI Conscientiousness, IPI Neuroticism, IPI Agreeableness, and IPI Conscientiousness are more strongly correlated with the criteria than are the rest of the predictors. Given the degree of interrelatedness among the Conscientiousness, Agreeableness, and Neuroticism predictors, relatively consistent results across criteria are expected. As we predicted, based on the Barrick and Mount (1991) meta-analyses, both Conscien- tiousness factors are correlated significantly across criteria. What was not predicted is that Neuroticism and Agreeableness are correlated as well or better with the various criteria. Because the predictors were correlated with each other and because a single selection decision must be .based on a combination of these predictors, we also conducted the various regression analyses described below. Because the various criteria are also correlated, our separate regressions for each dependent variable are at least partially redundant.


TABLE 4 Validities for Each of the Ten Personaliy Based predictors

Predictors MMPI Neuroticism MMPI Extraversion MMPI Openness MMPI Agreeableness Mh4PI Conscientious IPI Neuroticism IPI Extraversion IPI Openness IPI Agreeableness IPI Conscientious

PR

-.19* - .07 -.11 -.01 -.09 -.15* -.01 - .09 -.19* -.21*

PE -.16 - .04 -.19*

.06 -.17* -.22* -.12 -.04 -.17* -.17*

cc -.19* - .09 -.17* .04

-.lo -25' - .02 -.04 -.23* -.22*

TRT -.24* -.23* -.07 -.01 -.17* -a* -.16* -.15* -.19* -.14*

GPA -.B* -.17* -.15* .oo

- .20* --.21* -.04 .ll

-.30* -.18*

TRN

.27*

.21*

.ll

.05

.17*

.19*

.19*

.09

.17*

.09

-

Note: Sample sizes are as follows, Column 1 = 224, Column 2 = 145, Column 3 = 235, Column 4 = 219, Column 5 = 219, Column 6 = 303.

* p<.05

Regressions of Continuous Criteria

"hble 5 presents the results of hierarchical regressions of the continuous criteria on the various personality variables. In "hble 5, the IPI and MMPI variables were entered immediately after the Civil Service exam. Then, those scales (labelled Extra Scales) not judged to be indices of one of the "Big Five" factors were entered on Step 3 of the regressions.

Before describing the results presented in this table, it should be noted that the incremental validity of one inventory over another is not assessed in this table. Analyses with the IPI entered before and after the MMPI were conducted. Because these analyses took up valuable space and added little information to those that are now presented, they were removed and are available from the first author. The general conclusions suggested by the analyses that are not presented were the same as those suggested by the analyses described next.

The results presented in "hble 5 suggest several conclusions. First, the Civil Service exam correlates significantly with all five criteria, suggesting that there is a substantial cognitive component in early police performance and training academy success. As to the incremental validity of the personality measures, "hble 5 indicates that the IPI and MMPI add significantly to the prediction of Peer ratings, Counseling Card ratings, and GPA (above the Civil Service exam). As was mentioned above, the results for both personality inventories suggest that little is gained in terms of prediction beyond the Civil Service score, and that there is little evidence to suggest that both batteries need to be used to maximize predictability.

TABLE

5 H

iem

rchi

cal R

epss

wm

of C

riter

ia on

the

Civ

il Sa

ice

Exam

and

the V

ariou

s “B

ig F

ive”-

Base

d pre

dict

ors

-

Prob

atio

n Peer

coun

selin

g Fi

nal t

rain

ing

Grade

poin

t ra

tings

(N=2

24)

eval

uatio

m (N=145)

cards (N=235)

ratin

m (N=219)

aver

age (N=219)

pred

icto

rs

R2a

B

etab

R

2 B

eta

R2

Bet

a R

2 B

eta

2

Bet

a 1.

civil S

eMm

exam

.0

9*(.0

5)

.24

.10*(.09)

.32

.08*

(.08)

.29

.16*(.16)

.40

.38*(.38)

.62

2. P

I Neu

rotic

ism

.14

(.07)

.31*

.19*(.08)

-20

.15*(.08)

-.14

.u) (.12)

-.lo

.46*(.41)

.40*

rpI E

xtra

vers

ion

.02

-.22*

.10

-.04

.08

8 W

I

Neu

rotic

ism

-.31*

-.02

-.01

-.04

-:;* 8

PI O

penn

ess

- .02

.07

.18*

-.05

.14*

g

PI A

gree

able

ness

-.18

.02

-.18

.10

-.51*

;7 PI

Con

scie

ntio

us

-.13

.01

-.04

.01

WI

Ex

trave

rsio

n .13

.a*

.oo

-.06

z .07 9

-1

Ope

nnes

s .o 1

- .05

- .05

.10

WI

A

gree

able

ness

.07

.19*

.13*

.05

.10

I+

WI

co

nsci

entio

us

.04

-.13

.02

-.09

M

--.0

4 p

3. =a

Sc

ales

‘ .22*(.09)

.28

(.08)

.21

(.08

) .24 (.11)

.53*(.49)

L

Nore

: The

var

iabl

es la

belle

d IP

I Neu

rotic

ism--M

MPI

C

onsc

ient

ious

corr

espo

nd w

ith th

e “B

ig F

ive”

com

posi

tes f

rom

the

IPI a

nd M

MF’

I.

b B

eta w

eigh

ts fo

r eac

h in

divi

dual

varia

ble w

ithin

the

step

C

Bet

as fo

r ind

ivid

ual s

cale

s ar

e av

aila

ble u

pon

requ

est.

In th

e R’

co

lum

n, this in

dica

tes s

igni

fican

ce of

cha

nge i

n F

with

the

addi

tion o

f the

bloc

k, pc.10. I

n th

e Beta c

olum

n, th

is in

dica

tes s

igni

fican

ce of

for t

he e

ntir

e blo

ck o

f var

iabl

es w

ith C

attin

’s ap

prox

imat

ion o

f cro

ss-v

alid

ated

Ra

in p

aren

thes

es.

the B

eta w

eigh

ts, p<.lO.

t;

w


An examination of the various individual beta weights, however, suggests somewhat different conclusions. Beta weights suggest that, after partialling out the Civil Service exam, three predictors add to predictability better than any others; IPI Neuroticism, MMPI Agreeableness, and MMPI Neuroticism. It may be that a combination of these factors from the two inventories would be most useful.

In contrast to previous research, neither IPI nor MMPI Conscien- tiousness adds anything to the prediction of any of the criteria after the Civil Service exam scores are included in regression analyses. While the validities for Conscientiousness (see Bble 4) compare favorably to those reported by Barrick and Mount (1991), they are not as large as the validities for the Neuroticism and Agreeableness factors and all three overlap considerably. This combination of predictor intercorrelations and validities produced the suppressor effects involving IPI Neuroticism- Agreeableness which resulted in a significant positive beta weight for IPI Neuroticism-GPA (the validity coefficient was negative). Also, it should be noted that the validities from Barrick and Mount (1991) are not di- rectly comparable to those reported here. This is due to the fact that the Barrick and Mount (1991) validities were corrected for unreliability and range restriction while those presented here are not.

The final point to be made about this set of regressions is that, for probation ratings and GPA, the addition of the “extra scales” contributed significantly to prediction. This is, in large part, due to the contributions of the three MMPI Validity scales. The author-judges did not assign these scales consistently to any of the “Big Five” factors. In retrospect, it may be that the “low base rate” items that typically comprise validity scales could be incorporated into one of the existing “Big Five” personality factors or used to construct a sixth factor.

In an attempt to investigate the possibility that the validity scales could be incorporated into the “Big Five,” the original scale assignments made by the five authors of the present paper were revisited. It was found that the MMPI validity scales were most often assigned to the Neuroti- cism factor, though not often enough to conclude that they should be treated as measures of that factor. It was found that all three Validity scales were highly correlated with both Neuroticism factors, but also that the addition of the three scales increased internal consistency only for the MMPI Neuroticism factor (from .68 to .76). So it seems that “low base rate” items such as those that make up the MMPI Validity scales may be reahonable measures of Neuroticism.


TABLE 6 Hiemrchical L~gistic Regression of mmover on the Civil Service Exam

and the Kzrious “Big Five”-Based Predictors ( N =303)

Model chi-square predictors improvement Coefficient Standard error Wald

1. Civil Service exam 24.81. -.131 .027 23.09. 2. IPI Neuroticism 16.85. -.020 .021 0.74

IPI Extraversion .090 ,057 2.35

IPI Agreeableness .021 ,028 0.37 IPI Conscientious - .030 .030 1.06 MMPI Neuroticism .044 .024 3.19. MMPI Extraversion --.015 .032 0.21 MMPI Openness .009 ,028 0.11 MMPI Agreeableness - .056 ,060 0.86 MMPI Conscientious .052 .030 3.13.

Note: The variables labelled IPI Neuroticism-MMPI Conscientious correspond with

* p<.10

IPI Openness - .020 ,087 0.05

3. Extra scales 9.47

the “Big Five” composites from the IPI and MMPI.

Logistic Regression of Turnover

Hierarchical logistic regression was used to regress the dichotomous turnover variable onto the various predictors. ’bble 6 presents the results of these analyses.

The regression results for turnover are similar to those for the continuous dependent variables in two ways. First, the Civil Service exam contributes significantly to the prediction of turnover. Second, neither inventory stands out as superior to the other. Again, it should be noted that the incremental validity of one inventory over another is not assessed in this table. These analyses were conducted and are available from the first author. Briefly, these analyses revealed that, when the IPI block was entered first, it was significant while the MMPI block was not, whereas the opposite was true when the MMPI block was entered first. In several ways, however, the logistic regression results are unique.

The first difference is that, when the MMPI was entered before the IPI, none of the 10 principal predictors contributed significantly to prediction (though the MMPI factors as a block were significantly predictive of turnover) whereas, when the IPI was entered first, three predictors emerged as significant: IPI Extraversion, MMPI Neuroticism, and MMPI Conscientiousness. Of these, only MMPI Neuroticism stood out as significant in predicting the continuous criteria as well.


The second difference is that the extra scales do not contribute to the prediction of turnover as they do for probation ratings and GPA. In fact, given the degrees of freedom involved in adding the 10 extra scales to the prediction equation, a model chi-square improvement of 9.47 does not even approach significance.

Differential &diction

The possibility of differential prediction for minorities and women was addressed using moderated hierarchical regression (Cohen & Co- hen, 1983). After computing regression-weighted predictor composites for each combination of inventory and criterion (i.e., the sum of the weighted “Big Five” measures for each inventory and each dependent variable), each criterion was regressed on the predictor composites, gender (and race), and the interaction between the two in that order (for a total of 12 regressions for race and 12 regressions for gender). The Civil Service exam was not included in the computations of the regression weights for the predictor composites.

In the 12 regressions involving race, neither race nor any of the interaction terms were significant, suggesting that any differences between races in “Big Five” scores, from either the IPI or the MMPI, are reflected in actual performance ratings, GPA, and turnover. Hence, using the IPI and the MMPI would be fair to minority and majority groups using the Cleary definition of test fairness (“Uniform Guidelines,” 1978; Society for Industrial and Organizational Psychology Inc., 1987).

In the 12 regressions involving gender, however, four of the interaction terms were significant, suggesting that validities varied with gender. Three of the four significant interactions, however, involved the MMPI. Only one involved the IPI. Due to space constraints, the results of these analyses are not presented here but are available from the first author.

In an attempt to explore the nature of these interactions, the sample was broken down by gender, and validities for the two batteries were calculated for each gender. These analyses suggest that, for most of the criteria, the nature of these interactions is ~irnilar.~ For men, all six of the criteria are significantly predicted by the MMPI, and all but peer evaluations are significantly predicted by the IPI. For women, however, only probation ratings and turnover are predicted by the MMPI, and neither is predicted as well as it is for men. The IPI does better. In fact, the IPI predicts counseling cards and GPA better for women than it does for men. Overall, these analyses suggest that use of the IPI

3Plots of the interactions involving gender also supported the claim that the IPI results in less differential prediction than does the MMPI.


results in considerably less differential prediction than does use of the MMPI, although probation ratings, h a l training ratings, and turnover are predicted better for men than women by the IPI. Results of these analyses are available from the first author.

Discusswn

This study evaluated the predictive validity of two personality inventories as measures of the “Big Five” personality factors. The tests were not used in the selection of the participants, and the study was predictive. However, while the personality tests were not used in selection, interviews and the written Civil Service exam were used to screen applicants.

The present study set out to address four issues (a) the appropriate- ness of the “Big Five” taxonomy for understanding the relationship between personality measures and police performance, (b) the incremental validity of the IPI over and above cognitive ability and the MMPI, (c) the validity of personality measures across a variety of criteria, and (d) the possibility of differential prediction for minority and majority subgroups.

The fact that the author-judges were able to agree on scale assignments to “Big Five” factors provides some initial evidence that the IPI provides reasonable measures of Neuroticism, Extraversion, Agreeable- ness, and Conscientiousness while the MMPI provides measures of Neu- roticism, Extraversion, and Conscientiousness. However, several of the “Big Five” personality factors, as measured by the IPI and the MMPI, are highly correlated (>.60) with each other and moderately correlated (>.30) with the Civil Service exam, which is primarily a measure of cognitive ability. The correlations of “Big Five” factors across the two inventories were disappointingly low for Openness to Experience and Agree- ableness and to some extent for Conscientiousness. These results point to some problem with the “Big Five” taxonomy or with the assignment of the IPI and MMPI scales to the “Big Five” factors. Our results, however, appear consistent with one previous attempt to link specially developed measures of the “Big Five” to factor analytically derived dimensions of the MMPI (Costa et al., 1986). While this work developed some evidence for convergence between the “Big Five” factors and the MMPI factors, correlations were generally low and inconsistent across factors.

With respect to the incremental validity of the IPI, hierarchical regressions, with the inventories added into the equations as blocks, suggest that neither the IPI nor the MMPI adds much over Civil Ser- vice exam scores to the prediction of performance ratings, GPA, and turnover. One explanation for these relatively disappointing findings, however, is a lack of statistical power for some of these analyses. While


power was sufficient for detecting the effect sizes found for Counseling Card ratings (power = .86; &hen & Cohen, 1983) and GPA (power = .98), it was less than optimal for the other continuous variables (power = .69, .76, and .59 for probation ratings, peer evaluations, and final training ratings respectively). Future research should perhaps expect small effect sizes when examining the incremental validity of personality measures over measures of cognitive ability and choose sample sizes accordingly.

The validity of the Civil Service exam was particularly impressive since it was used to make recruit selections, producing direct restriction of range on the predictor. Personality measures may have been restricted indirectly because of their correlation with the Civil Service exam scores, but given the size of the correlation between the personality measures and the Civil Service scores, such restriction of range would have been minimal. In fact, the standard deviations of scores on the IPI were vir- tually identical to those reported for the norm groups in the test manual suggesting that no substantial restriction of range had occurred. Also of interest with respect to the Civil Service exam is the fact that it was significantly correlated with most of the “Big Five” measures. While these significant correlations are relatively small (mean T = -.23), they are contrary to previous research which has suggested that personality measures are unrelated to measures of cognitive ability. These significant correlations may be due to the fact that the panel interview was included in the Civil Service exam composite, and that the interview may be measuring something other than cognitive ability.

Individual beta weights for each “Big Five” factor suggest a more optimistic conclusion with respect to the personality inventories. The beta weights suggest that three factors consistently predict ratings and GPA: IPI Neuroticism, MMPI Agreeableness, and MMPI Neuroticism. Beta weights also suggest that three factors are significant predictors of turnover: IPI Ektraversion, MMPI Neuroticism, and MMPI Conscien- tiousness.

Our assessment of the validity of the personality measures across a variety of criteria showed that validities were highest for the measures of Conscientiousness and Neuroticism. In general, these scales best predicted GPA and Counseling Cards. Based on previous research (e.g., Barrick & Mount, 1991), the high validity for Conscientiousness was not surprising, but the high validities for the Neuroticism composites were. From a ratipnal standpoint, however, the validity of an adequate measure of Neuroticism in predicting performance of law enforcement agents seems reasonable. Another term for Neuroticism is Emotional Stability. Given the potential for extremely high stress on a daily basis in law enforcement, it is not surprising that Emotional Stability would


predict police performance. Future research in this area should further investigate the link between Neuroticism and police performance.

Finally, differential prediction analyses suggest that the use of the “Big Five” factors as measured by the IPI and MMPI produces differential prediction for gender but not for race with the “Big Five” predicting ratings, GPA, and turnover better for men than for women. This differential prediction, however, is more pronounced for the MMPI than it is for the IPI. This fact, when combined with the information from the validities (’Itrble 4) and the regressions (’Itrbles 5 and 6), suggests that the IPI measures of Neuroticism, Extraversion, Agreeableness, and Consci- entiousness may be the most appropriate personality-based predictors of police performance.

Perhaps the most serious limitation of the present study is the possibility that the subjects had a different motivational set than would actual applicants since the subjects had already been selected into the training academy. However, while this is certainly a possibility, given the fact that tests were administered on the first night of training, it does not seem likely that the subjects had been socialized in any meaningful way or that they were any more or less motivated than actual applicants. This limitation, nevertheless, remains a possibility.

REFERENCES

Azen SP, Snibbe HM, Montgomery HR. (1973). A longitudinal predictive study of success and performance of law enforcement officers. Journal of Applied Psychology, 57, 1w192.

Barrick MR, Mount MK. (1991). The “Big Five” personality dimensions and job performance: A meta analysis. PERX~NNEL ps~mouxiy, 44,l-26.

Bartlett CJ, Bobko P, Mosier SB, Hannan R Jr. (1978). =sting for fairness with a moderated multiple regression strategy: An alternative to daerential analysis. PERSONNEL ps~mouxiy, 31,233-242.

Butcher JN. (1979). Use of the MMPI in personnel selection. In Butcher JN. (Ed.). New developments in the use of the MMPI. Minneapolis, MN: University of Minnesota Press.

Cohen J, Cohen F! (1983). Applied multiple regmswnlcomhtion for the behavioral sciences. Hillsdale, NJ: Erlbaum.

Colarelli NJ, Siege1 SM. (1964). A method of police personnel selection. Ihe Journal of Criminal Law, Crimimlogy, and Police Science, 55,287-289.

Costa PT Jr, Busch CM, Zonderman AB, McCrae RR. (1986). Correlations of MMPI factor scales with measures of the five factor model of personality. Journal of honaliry Assessment, 50,640650.

Costa PT Jr, Zonderman AB, McCrae RR, Williams RB. (1985). Content and comprehen- siveness in the MMPI: An item factor analysis in a normal adult sample. Journal of &rsona& and Social Psycho&, 48,925-933.

Digman JM. (1990). Personality structure: Emergence of the five-factor model. Annual Review of Psychology, 41,417440.

Gottesman J. (1969). Personaliry patterns of urban police applicants as measmd by the MMPI. Hoboken, N J Stevens Institute of Technology.


Harris MM, Schaubroeck J. (1988). A meta-analysis of self-supervisor, self-peer, and peer- supervisor ratings. PERSONNEL psmouxiy, 41,43-62.

Hmke JF, Krauss HH. (1971). Personality characteristics of highly rated policemen. PER- s o " ~ ~ p s ~ c ~ o u x i y , 24,679-686.

Inwald R. (1988). Five-year follow-up study of departmental terminations as predicted by 16 preemployment psychological indicators. Journal of Applied Psychology, 73, 703-710.

Inwald R, Knatz H, Shusman E. (1983). Znwald Personality inventory manual. New York Hilson Research.

Inwald R, Shusman E. (1984). The IPI and MMPI as predictors of academy performance for police recruits. Journal of Police Science and Administmtion, 12,l-11.

Johnson JH, Butcher JN, Null C, Johnson KN. (1984). Replicated item level factor analysis of the full MMPI. Jowral of Personal@ and Social Psychobgv, 49,105-114.

Mass G. (1979). Using judgment and personality measures to predict effectiveness in police work An exploratory validation study. Dissertation Abstmcts International, l980,5063,40B.

McAllister L. (1977). The MMPI as a selection instrument for law enforcement officers in Dakota County, 1969-77. Presentation at Police Officer aaining Board Seminar. Willmar, MN.

McCrae RR, Costa F'T Jr. (1985). Updating Norman's "adequate taxonomy": Intelligence and personality dimensions in natural language and in questionnaires. Journal of Personaliiy and Social Psychology, 49,710-721.

Murphy JJ. (1972). Current practices in the use of psychological testing by police agencies. Journal of Criminal Law, Criminology, and Police Science, 43,570.

Norman WT. (1%3). Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality ratings. Journal of Abnormal and Sock1 Psychology, 66,574-583.

Nowicki S. (1966). A study of the personality characteristics of successful policemen. Police, 10,39-41.

Sacuzzo DP, H i e s G, Lewandowski D. (1974). Program for psychological assessment of law enforcement officers. Psychological Reporis, 35,651654.

Seashore SE, Indik BP, Georgopoulos BS. (1960). Relationships among criteria of job

Schmitt N, Gooding RZ, Noe RA, Kirsch MF! (1984). Meta-analyses of validity studies published between 1964 and 1982 and the investigation of study characteristics. pERso"ELPsYcH0LoGY, 37,407422.

Schneider B, Schmitt N. (1986). & f i g organizations. Glenview Illinois: Scott, Foresman. Society for Industrial and Organizational Psychology, Inc. (1987). hincipks for the valida-

tion and use of selectionprocedures. College Park, MD. Author. Uniform guidelines on employee selection procedures. (1978). FederaIRegirer, 43,382%

38315. U.S. Congress, office of lkchnology Assessment. (1990). The use of i n t e e tests forpre-

employment screening, (OTA-SET-442) Washington, D C US. Government Print- ing Office.

performance. Jowtnl OfApplied Psychology, 44,195-202.

THE "BIG FIVE" PERSONALITY FACTORS IN THE IPI AND MMPI: PREDICTORS OF POLICE PERFORMANCE

Documents

Transcript of THE "BIG FIVE" PERSONALITY FACTORS IN THE IPI AND MMPI: PREDICTORS OF POLICE PERFORMANCE