The Effect Of Interviewer Attitude on Survey Cooperation

Bulle/ill de llte/hodologie Sociologique (59 rue Pouchet, F 75017 Paris), October 2002, N. 76 - pp. 27-44

THE EFFECT OF INTERVIEWER ATTITUDEON SURVEY COOPERATION

by

Michael Lemay(Department of Sociology, Universite de Montreal)

Claire Durand(Department of Sociology, Universite de Montreal;

[email protected])

Resume - L'effet de I'attitude de I'intervieweur sur la cooperation lors d'une enquete : Larecherche pour ameliorer la qualite des enquetes par questionnaire montre que des facteursmethodologiques ont une influence significative sur la cooperation des repondants. Le recherchepresentee ici est inspiree de la sociologic du travail et des etudes de I'impact des altitudes desintervieweurs concernant leur motivation et ('''auto-efficacite'' pour convaincre les repondants aparticipcr a l'enquete . Une enquete aupres des intervieweur s travaillant pour trois etabli ssernentsprives d 'enquete a ete menee pendant la campa gne pour les elections federate s canadiennes ennovernbre 2000. Les resultat s montrent qu'il existe des liens entre la motivation, le sens d"'autoefficacite" et des comportements associes, d'un cote, et des performances des intervi eweurs, del'autre. Cependant, certains resultats peuvent etre mieux expliques par les difference s de formationdes interview eurs. De plus, l' absence d'une mesure plus fiable des performances peut aussiexpliquer la faiblesse des resultats, D'autres etudes doivent examiner les mesures de performancepour comprendre les changements dans les performances avec Ie temps. Enquetes parquestionnaire, Enqueteurs, Performances des enqueteurs, Attitudes des enqueteurs, Mesuredes performances d'enquHeur.

Abstract : Research to improve the qualit y of social surveys has shown that methodological factorshave a significant impact on respond ent cooperation. The research presented here is inspired by thesociology of work and studies the impact of interviewers' sense of motivation and "self-efficacy" inconvincing respondents to answer surveys . To do so, a survey of interview ers workin g for threeprivate pollsters was conducted during the Canadian federal election campaign of November 2000 .The results indicate some evidence of relationships between motivation, sense of self-efficacy (andassociated behaviour) , and interviewer performance. !lowever, some of the results could be betterexplained by difference s in training. In addition, the lack of a better measure of performance mayalso explain the weakness of the results. Future studies should examine performance measurementand try take into acco unt how performance changes over time. Surveys. Interviewers, InterviewerPerformance, Interviewer Attitudes, Measurement of Interviewer Performance.

27

at UNIVERSITE DE MONTREAL on April 24, 2015bms.sagepub.comDownloaded from

http://bms.sagepub.com/

p. 28 - Bulle/ill de Me/hodologie Sociologique (59 rue Pou cher, F 750 17 Paris), October 2002 , N. 76

CONTEXT

For as long as social surveys have been used , it has been thought that interviewershave an impact on the quality of the data they collect. Scores of studies have tried toassess the causes and consequences of various biases by looking at the interviewercharacteristics, behaviour, or attitudes. This research builds on these studies andalso adds new perspectives.

The general notion behind this study is that an interviewer can be thought of as anyother salaried employee. Thi s starting point affords a new point of view, one fromwhich sociological literature gains relevance.

Two perspectives on interviewing. Broadly speaking, there are two contrastingviews on how the work of the interviewer should be considered . These views ariseout of a fifty-year-old debate among researchers in the disciplines of sociology andhuman resources management concerning the differences between formal andinformal work management and the sources of work motivation. The science ofwork management, or "scientific management," was developed by Frederick Taylorat the beginning of the 20th century . The main tenet s of scientific managementwere a division of work into small tasks, an optimization of each of these tasks, anda salary based on piece work . It aimed at controlling workers to ensure a high levelof productivity and a certain product uniformity. In the 1960s, a new school ofthought, called the "human relation movement", offered an alternative to Taylor'stheory. It regarded work as more than simply the production of goods or servicesand stressed the distinction between the formal (i.e., planned by the organization)and informal (e.g., interpersonal relationships) aspects of the workplace. Moreover,it considered that economic incentives could not, by themselves, explain anindividual 's motivation to work .

The equivalent of this debate appears in the literature on survey methodology.Some researchers, empha sizing the scientific nature of the surveys, promote a ratherstrict adherence to the script , leaving no place for initiative by interviewers. Theyargue that any misunderstandings which occur during an interview can be clarifiedby non-directive probing . Also , they consider the best way to reduce bias and avoidthe need for probing is to improve questionnaire design (Fowler and Mangione,1990).

On the other hand , some researchers argue that interviewers should be given moreleeway . They offer proof that, under certain circumstances, a non-standardizedinterviewing style can produce more accurate results than full adherence to theinterview script and question wording (Schober and Conrad, 1997). They have alsoshown that when interviewers tailor the survey introduction, there is a positiveimpact on respond ent willin gness to cooperate (Morton-William, 1993; Van denBerg and Houtkoop-Steenstra , 1999; Deleeuw, 1999).

Interviewer tusks : COI1l'ince and intervi ew. Schematically, interviewers mustperform two different tasks, in which they play specific roles : they must act as a

28



Bulletin de ~Uthodologie Sociologique (59 ruc Poucher, F 750 17 Paris), October 2002 , N. 76 - p. 29

petitioner to convince respondent s to participate in the survey, and they must alsoplay their "real" role of interviewer , asking the questions and recording theanswers.

The word petitioner, used by Morton-William (1993) to describe the role of theinterviewer, expresses very clearly what the presentation of the survey and thecontact with the selected respondent are all about. The interviewer asks for a favourand therefore is not in a position of authority. The extent to which interviewers arecomfortable with this aspect of the work might affect their ability to convincerespondents and, ultimately , interviewer performance . The goal of this research isto examine these hypotheses by looking at some of the key elements in thepsychology and sociology of work related to the study of interviewer performance.The study focuses on the petitioner role (i.e., on the ability of interviewers toconvince). Performan ce is defined accordingly .

Motivation. As with any other job , motivation toward different aspects of the workcould play a significant role in interviewer performance. There are two differenttypes of motivation. Intrinsic motivation refers to the satisfaction a worker getsfrom accomplish ing the task itself. The desire to keep the job stems from how aworker feels toward the task itself, how useful they see it, and how successful theyare in performing it. External incentives such as wages and fringe benefits areconsidered to be weaker motivators than enjoyment of the task itself (Herzberg etaI., 1959) suggested that motivation and positive feelings toward the task may wellhave an impact on worker performance. However, most of the existing research hasfound no relationship between satisfaction and performance (Feertchak, 1996;Frances, 1987).

Herzberg et al. (1959) were among the first to propose that "job enrichment" couldincrease worker motivation and satisfaction . This is similar to what somesociologists call the "quality of work" approach (Lowe , 2000) . It consists in givingworkers constructive feedback to help improve their work and in providing moreopportunities to take initiatives , assume responsibilities, and influence decisionmaking. In other words, job enrichment provides interesting work that appea ls toworkers' abilities, knowledge, and intelligence . By instituting such a policy,employers may see an increase in performance and productivity or, at the very least,lower turnover. The literature on survey methodo logy says very little about how toimprove quality by means other than monetary incentives and close supervision.

Knighton, Hubbard, and Michaud ( 1999) examined how interviewer attitudeaffected survey cooperation in a study of Statistics Canada interviewers conductinga face-to-face survey. The questionnaire used by the interviewers included items onattitudes toward survey cooperation. Three attitudes were significant ly related to theresponse rate: the interviewers who believed that even the most reluctantrespondents could be persuaded to participate tended to achieve higher responserates; those who believed that it is better to withdraw and try again later if arespondent appears likely to refuse also achieved higher rates; on the other hand,the interviewers who believed in emphasizing the voluntary nature of the surveyseemed to get lower response rates. Five of the items on the Knighton et al.

29



p. 30 . Bulletin de AUtllOd%gie Sociotogique (59 rue: Poucher, F 75017 Paris) , October 2002 , N. 76

questionnaire were used in a previous study by Lehtonen (1996), who obtainedsimilar results : the interviewers who were more in favour of persuasion had higherresponse rates .

Therefore, it is hypothesized that : (III) Interviewers who have positive attitudestoward persuasion tend to achieve better performance.

Self-efficacy and attribution. Though interviewers are not the only aspects of thesurvey process that can affect respondent participation, they are the ones in directcontact with the respondents. They are expected act in such a way as to maximizerespondent participation. In other words, they arc expected to control everysituation, or at least be as effective as possible.

This expectation is connected with the notion of "self-efficacy," or the belief inones abilities, as elaborated by Albert Bandura over the past thirty-five years.Individuals have a perception of their ability to perform a given task or handle agiven situation. The success they foresee in their performance can influence thetype of activities they get involved in and the time and effort spent on that activity.Those who perceive their chances of success as being good are more likely topersevere at a given task . Self-efficacy can vary from situation to situation, aproduct of one 's perception of both the situation and ones self. This notion isintertwined with that of attribution. People who feel that internal factors are thecause of a given situation will be 1I10re likely to think they can act upon it, which, inturn, could improve their performance.

To what extent interviewers see themselves as the key factor in survey cooperationis possibly a very important element, one that could affect their motivation andeffectiveness. Individuals who believe they have the power to affect or change asituation are more likely to act upon it than those who feel they are restrained byforces beyond their control. The interviewers, who believe they are the maindeterminants of survey participation, may be more likely to succeed compared tointerviewers who believe that the main causes of survey non-response have more todo with the respondents themselves, the orientation of the survey firm, and otherexternal factors such as questionnaire quality.

Therefore, it is hypothesized that : (1-12) Interviewers who attribute surveycooperation to their own behaviour rather than to external factors tend to performbetter.

Morton- William (1993) shows that interviewers who adapt the survey introductionto respondents and their situations get higher response rates. These interviewersprefer to clear up respondents' reservations instead of engaging in the selectionprocess. They will also offer to call back at a better time rather than risk a strongrefusal. Similar tailoring strategies were found to be significantly related to theinterviewer response rate (DeLeeuw, 1999). Van den Bergh and HoutkoopSteenstra (1999) also found higher response rates with interviewers who used anon-standardized form of introduction. Furthermore, other researchers have foundthat interviewers rarely use the arguments in the interview script to convincerespondents.

30



Bulletin de ~Uthodologie Sociologique (59 rue Poucher , F 75017 Paris), October 2002, N . 76 - p. 31

Consequently , it is hypothesized that: (H3) Interviewers who tailor the introductionto the respondent's situation will achieve better performance.

Characteristics . There is very little evidence that interviewer characteristics, suchas age, gender, and education, affect their ability to convince respondents (Hox,1999). Thus, it is hypothesized that personal characteristics have no effect.However, time (i.e., seniority and experience) is thought to have a positive effect oninterviewer performance. Therefore, (114) Personal characteristics of interviewerssuch as age, gender, and education do not affect their ability to convincerespondents to participate. However , (liS) Experience and seniority have a positiveeffect on performance.

METHODOLOGY

Population. I Interviewers from three private polling firms were asked to take part inthe study during the Canadian federal election campaign of November 2000. Theeligibility criteria were: I) they had to be working on an electoral survey (thiscondition was imposed to ensure a certain uniformity in the type of surveys and,thereby, the context of the performance); 2) they must have worked a minimum oftwenty hours (in the case of the newly recruited interviewers); and 3) they had to beable to answer the questionnaire in French since, at the time of the study, thequestionnaire was only available in that language. Two of the firms had Frenchspeaking interviewers while the third had both French and English-speakinginterviewers (thus, in this latter firm, only a minority of interviewers participated inthe survey). A total of forty-five eligible interviewers participated in the study .Thirty-nine of them provided the code that allowed a match to be made with thefirm's databases.2 Little difference in performance was found between interviewerswho participated in this study and those who didn't. In two of the firms, theperformance of participating interviewers tended to be higher, which is explainedby the fact that under-performing interviewers may not have stayed long enough tobe considered eligible. In the other firm, interviewers who did not participate alsohad the particularity of being English-speaking, making it difficult to cancel out theeffects.

Measurement . Measures of attitudes, behaviour related to survey interviewing, andcharacteristics of interviewers were based on the data gathered with a selfadministered (PAPI) questionnaire. In addition, a measure of performance wasdevised using the information available in the firms' databases.

Independent variables . Intrinsic motivation was specifically measured by positiveattitudes toward persuasion. Five measures of persuasion were used: two scales(POSPERS and NEGPERS) comprising six and three items respectively' (!l=0.7472and 0.6196), and three individual questions about interviewers' comfort withpersuasion and the behaviour they felt to be appropriate with both respondents ingeneral and also with refuscrs ."

31



p. 32 • Bulle/ill lie AlhhodologieSociolugique (59 rue Poucher, F 7501 7 Paris), October 2002 , N . 76

Self-efficacy was measured by fourteen questions or scales concerning attitudes(mostly attributions) and twelve questions on the actual behaviour associated withthese attributions.

For attitudes, sel f-efficacy was measured by four scales (VOCAL, INITIAL, PSY,SUI'MORAL) and ten individual items focussing on the perceived determinants ofsurvey cooperation and on factors 6 seen as affecting the interviewers' ability toconvince respondents to participate.

Ten of the twelve behaviour questions measured the self-reported frequency oftailoring the survey introduction (according to respondent characteristics or factorssuch as age, education, accent, stres s, or mood) and the self-reported frequency ofusing certain arguments to elicit cooperation (four individual items /and a three-itemscale ~BENEFICE) referring to arguments emph asizing the usefulness of thesurvey ). The survey also assessed intervi ewing style using a seven-point scaleranging from authoritative to conciliatory and from friendly to reserved.

Finally, basic demographics (age , language, education); and questions on the typeand durati on of previous experience, the number of work hours, and seniority wereused to explore the fourth and fifth hypotheses.

Dependent variahle. The call management databases were used to develop ameasure of performance. These databases contained the results of every call attemptfor each survey; the code of the interviewer responsible for each call; and relatedinformation such as date, time , and interview duration. A good index of interviewereffectiveness must exc lude all non-contacts and focus only on the telephonenumbers that are known to be valid 9 and eligible.!" Thi s is because for interviewersto use their persuasion skill s, they must actuall y speak to a person who is eligible toanswer the survey . Failing to take this requirement into account would bias theindex toward interviewers who work more during periods where more people are athome and reachable. To take this into account, Loosveldt, Carton, and Billiet(1999) computed a contact rate (the proportion of valid and eligible units) and asuccess rate (index of interviewer performance), and Knighton, Hubbard, andMichaud (199 9) made a distinction between non-response due to refusal and nonrespon se due to other cau ses involving eligibility criteria (language, illness, etc).However, both studies were conducted in the context of face-to-face interviews; thecomputation of a non-biased index in the context of computerized telephonesurve ys must take into account the fact that telephone numbers are not ascribed toany interviewer in parti cular. Some interviewers specialize in calling back toconvince households that have previously refu sed to cooperate. To usc all thecontacts in the computation of the index would thus penalize interviewers assignedto refusal conversion because refusers are, by definition, harder to convince.

Therefore, a first-contact cooperation rate was created: only the call s in which thefirst contact with a household occurred were selected, and the ratio of "completes"to "completes plus refusals" was calculated for each interviewer. I I A nonparametrictest , s imilar to ANOVA , called Kruskall-Wallis H.,12 is used to examine therelationship betwe en independent variables measured at the ordinal or nominal

32



Bulletin de Mithodologie Sociologique (59 rue Pouchct, F 75017 Paris), October 2002 , N. 76 - p.33

Icvel and this performance index. In the case of continuous variables, Pearson 'scorrelation was used.

Since the sample size is small, a p value of .10 or less was used as a generalindication of the "rarity" or "potential" of the relationships. Sinee no sample wasdrawn, inference to a general population is not theoretically appropriate.

RESULTS

Motivation. The first group of hypotheses refers to the effect of interviewermotivation on effectiveness. Of the five variables used to measure intrinsicmotivation, three were related to cooperation rate. Figure 1 shows a generalincrease in the mean cooperation rate as the positive attitudes toward persuasion,measured by the POSPERS scale (favourable attitude toward persuasion), increase(F0.310, p=0.070) . From a mean value in the 25 to 50-percent range for theinterviewers who have less favourable attitudes toward persuasion, the meancooperation rate increases to between 60 and 70 percent for interviewers who havemore favourable attitudes . See Figure 1 - Mean Cooperation Rate According toAttitude Toward Persuasion .

In addition, interviewer cooperation rates vary with two indicators of their preferredstrategy with respondents. Figure 2 shows that the interviewers who believe it isbetter to seek immediate cooperation than to arouse the respondent's interest in thesurvey tend to achieve better cooperation rates (56.4 versus 45.6; H=3.174;p=0.075). See Figure 2 - Mean Cooperation Rate with Respect to Preferred Strategywith Respondents in General and with Refusers

Likewise, cooperation rates vary depending on the strategy favoured for refusalconversion. Those who believe that the same interviewer should call back in case ofa refusal have higher cooperation rates (H=5.516; p=O.O 19).

Despite the fact that two variables were not related to cooperation rate ("comfortwith persuasion" and "negative attitude toward persuasion," NEGPERS), the resultsoffer some support for the hypothesis that interviewer motivation affectsperformance. The interviewers who have a more positive attitude and motivationtoward the persuasive aspect of the work tend to perform better.

Self-efficacy. The second group of hypotheses deals with perceived self-efficacy. Ofthe twenty six indicators used to explore this group of hypotheses, only four wererelated to the cooperation rate at first contact. Two of these items focus on theperceived determinants of survey cooperation . See Figure 3 - Mean CooperationRate According to Perceived Determinants of Survey Cooperation.

Figure 3 shows that interviewers, who believe that questionnaire length (H=4.012,p=0.045) or survey sponsor reputation (1-1=2 .756, p=O.097) influence potentialrespondents' willingness to take part in the survey, tend to achieve lower

33



p.34 - Bulletin de Mhhmlologi e Socio logi que (59 rue Pouchct, F 75017 Paris), October 2002, N. 76

cooperation rates . This means that interviewers who tend to allribute respo ndentcooperation to external factors achieve lower performance.

Two items related to the self-reported behaviour of interv iewers were alsopositively related to cooperation rate. Interviewers who report tailo ring the surveyintroduction according to the respondent's level of stress (Figure 4.) get a highe rcooperat ion rate than those who do not adapt the introduc tion (52 .6% vs 38.0%;H=5.402 ; p=0.020). See Figure 4. Mean Cooperation Rate Acco rding to Selfreported Tailoring Depending on Perceived Stress Level of Respo ndent.

The second finding related to interviewer behaviour was that those who see theirinterviewing style as rather autho ritative tend to get a higher cooperation rate.Figure 5 shows an increase in cooperation rate along with an increase in perceivedlevel of authoritativeness (r=0.286; p=0.086) . See Figure 5 - Mean Coo peratio nRate Accord ing to Self- reported Interviewing Style.

Severa l variables used to exp lore the self-efficacy hypotheses do not appear to berelated to cooperation rate. For example, it does not mailer whether the intervieweremphasizes the usefulness of the survey , the limited time requir ed for the interview,or the interes t of the survey topic. Nor is tailor ing acco rding to gender, mothertongue, or mood related to cooperation rates. Tailoring acco rding to age appears tobe significant; however, a closer exam ination shows that 36 out of 37 interviewersoften or sometimes tai lor the introduction accor ding to the respondent' s age .Therefore, this behavio ur can bc considered commonplace and does notdifferentiate between interviewers. No relations hip with cooperation rate was foundwhen those who often tailor according to age were compared with the others.

In addition, severa l potentia l determinants of survey coo peration had norelation ship with cooperation rate. There was no differe nce in effec tivenessbetween interviewers who believed that factors such as emo tions, interest in thesurvey topic, call circumstances, and questionnaire qua lity have an influence andinterviewers who did not think such factors affect survey cooperation. Therefo re,the data used here only partially support the self-efficacy hypothesis.

Characteristics. As hypothesized, persona l characteristics were not related tocoope ration rate. Interviewer age, gender, mother tongue, language spoken at home,and education level had no relationship with ability to convince respondents.

However, some variab les measuring expe rience appea r to be related to interviewereffectiveness . Interviewe r cooperation rate at first contact was related to the numberof hours worked per week (r=0.324, p=0.054) and the amount of experience withcurrent employer (r=0.292, p=O.084). Cooperation rates were not related toexperience in call centres, the number of polling firms the interviewer had workedfor, or the amount of experience as a survey interviewer.

34



Bulletin de Method%gie Soctotogique (59 rue Pcuchct, F 75017 Pari s), October 2002 , N. 76 - p. 35

DISCUSSION

The results presented above are consistent with previous research. Interviewers whoexpress a higher level of motivation toward persuasion of respondents (the intrinsicaspects of interviewing) tend to be more effective in getting respondents tocooperate . These results are consistent with Knighton et al.(1999) and Lehtonen(1996), although the methods used are different. For example, Knighton et at.(1999) did not create a scale, only the individual variables were used. Two of thethree significant relationships point in the same direction as the results found in thepresent research: the more interviewers express attitudes in favour of persuasion,the better they perform.

Only a few of the indicators of self-efficacy are related to performance.Interviewers who do not attribute their performance to external factors and whotake control of the interview situation tend to perform better. Interviewers whoreport tailoring the introduction according to perceived stress level tend to gethigher cooperation rates. Morton-William (1993), DeLeeuw (1999), Van den Bergand Houtkoop-Steenstra (1999) have found similar results, though all used differentmethods in different contexts. For example, it should be pointed out that it is thereported /ailoring of the introduction that is the focus of this discussion, not actualbehaviour observed by a third party, as was the case in Morton-William (1993).Furthermore, Van den Berg and Houtkoop-Steenstra (1999) used an experimentaldesign in which different groups of interviewers were asked to use introductionswith different levels of standardization.

Only one of the five items related to tailoring-the one pertammg to therespondent's perceived stress-was related to cooperation rates. The other items,more readily available to the interviewers, such as gender, age, and accent (as aproxy for mother tongue), were not related to performance. This implies that themost effective tailoring (i .e., that affects cooperation rate) is done according to therespondent's situation rather than to some stereotype associated with therespondent's characteristics. During the social interaction between interviewer andrespondent , attitudes and behaviour appear to be of more importance thancharacteristics, which is consistent with the general context of this research.

Personal characteristics were not related to performance. This finding is consistentwith the literature (Hox , 1999). However, interviewer cooperation rates increasedwith seniority within the firm and number of hours worked per week. This variablecan be seen as a measure of the interviewer's experience on the project. There aretwo ways to explain this finding: either interviewer performance improves asexperience with the questionnaire increases, or interviewers who perform better arein higher demand and , as a result, work more hours.

One question should be raised . The items on attitudes toward persuasion may not bemeasuring what they are supposed to be measuring. They may, in fact, measurewhat interviewers have been taught during training rather than what they personallybelieve . There could be a mediating effect of training or firm . For instance,

35



p. 36 - Bulle/in de ~/;'lwdologie Sociologique (59 rue Puu chet, F 750 17 Paris), October 2002, N. 76

interviewers, who reported having received trammg on how to avoid refusals,appear to be more in favour of persuasion (H=2.734 , p=O.098) and to have a moreauthoritative interviewing style (11=3.747, p=O.053) .

Whether comparing the respondents alone (H=3.l74, p=O.205) or all of theinterviewers in the field management databases (H=O.347, p=O.556), there was nodifferen ce between firms in the mean cooperation rate of interviewers. However,differences were found from firm to firm regarding some of the attitudes andbehaviour measured . In some firms, the interviewers were more prone to usingarguments that emphasized the usefulness of the research and the foreseeablepositive impact on society. They expressed more positive opinions about persuasionand reported tailoring the presentation of the survey more often.

Multivari ate analyses, which would have been helpful to resolve this apparentcontradi ction , could not be used due to the small number of cases, a problemcommon to research on interviewers, as stated by Sudman, Bradburn, Blair , andStocking (1977). Finally , the cross-section al nature of this study makes itimpossible to establish the direction of the relationships that were found .

Despite its limits, this study shows some support for the importance of taking taskrelated motivation into consideration when it come s to predicting interviewerperformance. The results also empha size the necessity of allowing some initiativeon the part of the interviewer. This is a matter of reduc ing job turnover andincreasing data quality in an era of declining response rates .

Future studies should allow for the analysis of the evolution of performance. Theyshould also allow for the further exploration of the attitudes and behaviour that maybe related to interviewer performance and, more specifically, those related tointerpersonal competence and self-effi cacy. However, one must raise the possibilitythat the absence of a relationship between certain attitudes and performance is dueto the quality of the measure of performance itself. Is cooperation rate at firstcontact the best measure of performance in telephone surveys or should a morerefined measurem ent, one that takes into account the complete work load ofinterviewers, includin g refusal conversion , be devised ?

NOTES

I . The interviewers were not sampled. All interviewers meeting the eligibilitycriteria were asked to answer the survey. The main consequence of using a strategythat focuses on a population rather than a representative sample is that it isimpossible to generali ze the findings beyond the group specifically studied.

2. The analysis included 37 of these 39 interviewers. Two interviewers who hadworked only on the second phase of a panel survey could not be included becausecontacts for the second phase do not formall y qualify as "first contacts," which wasthe focus of the anal ysis.

36



Bulletin de hUthodologie Soctotogtque (59 rue Poucher, F 750 J7 Paris). October 2002, N. 76 • p. 37

3. Cf.: Appendix A for the content of these scales.

4. For example, persuading a respondent or accepting a refusal if felt that theanswers given by the respondent would not be accurate; interesting the respondentin the surveyor seeking a quick decision on participation; different or sameinterviewer in case of refusal conversion.

5. Cf.: Appendix A.

6. Call circumstances, field supervision, survey topic, survey sponsor, firm, moralsupport, etc.

7. Not a telemarketing call , every opinion counts , very short interview, means tohelp the interviewer.

8. Cf.: Appendix A for the detailed composition of the scale .

9. For example, other than business or a fax/modem lines.

10. That is households where at least one person is eligible to answer the survey .

II. A similar index was used by Link and Armsby (2001) (ratio of refusals tocompletes).

12. An alternative to analysis of variance (ANOY A) that is used whenever theassumptions of the latter cannot be met (Blalock , 1979, p.248) . The small numberof cases available in this study made it impossible to meet the criteria of a normaldistribution .

REFERENCES

Blalock, Hubert M. (1979): Social Statistics, Second Revised Edition . McGrawHill, chapters 14 and 16.

DeLeeuw, Edith (1999) : "How do Successful and Less Successful InterviewersDiffer in Tactics for Combatting Survey Nonresponse?" Bulletin de MethodologieSociologique 62, 29-42 .

Feertchak, Helene (1996): Les motivations et les valeurs en psycho-sociologie."Cursus" collection, "Psychologic" series . Paris: Armand Colin, chapter 14, 144156.

Fowler Jr., Floyd J., and Thomas W. Mangione (1990) : Standardized SurveyInterviewing : Minimizing Interviewer-related Error. Newbury Park : Sage .

37



p. 38 - Bultetin tie MhllOdologie Soctotogtoue (59 rue Poucher , F 75017 Paris), October 2002 , N. 76

Frances, Robert (1987): "La motivation au travai1." Traite de psychologie dutravail . Claude Levy-Leboyer and Jean-Claude Sperandio (eds) . Paris: Pressesuniversitaires de France , 347-355.

Herzberg, Frederick, Bernard Mausner, and Barbara Bloch Snydernam (1959): TheMotivation to Work, Second edition. New York: Wiley.

Hox, Joop (1999): The Influence of Interviewer'sHousehold Survey Nonresponse: An InternationalConference on Surveyhttp ://www .jpsm.umd.edu/icsn/main/abstractsList .htm.

Attitude and Behavior onComparison. International

Nonresponse.

Knighton, Tamara, Jennifer Hubbard , and Sylvie Michaud (1999): InterviewerAttitudes and Response Rates: Implications for the National Longitudinal Survey ofChildren and Youth (NLSCY), Statistics Canada. International Conference onSurvey Nonresponse, Portland, Oregon.

Lehtonen , Risto (1996) : "Interviewer Attitudes and Unit Nonresponse in TwoDifferent Interviewing Schemes." International Perspectives on Nonresponse,Seppo Laaksonen (ed). Helsinki : Tilastokeskus, 130-140.

Link, Michael W., and Polly P. Armsby (2001): Predicting Persistence andPerformance among Newly Recruited Telephone Interviewers. AmericanAssociation for Public Opinion Research, 56th Annual Conference, Montreal,Quebec, May 18-20, 2001.

Loosveldt, Geert, Ann Carton , and Jaak Billiet (1999): Evaluation of theRelationship Between Data Quality and Interviewer Behavior during DoorstepContact in Face-to-Face Interviews, International Conference on SurveyNonresponse. Portland, Oregon, October 28-31, 1999.

Lowe, Graham S. (2000): The Quality of Work: A People-Centred Agenda, Oxford :Oxford Univerity Press.

Morton- William, Jean (1993): Interviewer Approaches. Dartmouth: Aldershot.

Schober , Michael F., and Frederick G. Conrad (1997): "Does ConversationalInterviewing Reduce Survey Measurement Error?" Public Opinion Quaterly 61,576-602.

Sudman , Seymour, Norman M. Bradburn, Ed Blair, and Carol Stocking (1977):"Modest Expectations: The Effect of Interviewer Prior Expectations on Response."Sociological Methods and Research (6),177-182.

Van den Bergh, Huub, and Hanneke Houtkoop-Steenstra (1999) : Effects of Style ofIntroduction on Nonresponse. International Conference on Survey Nonresponse,Portland, Oregon , October 28-31 , 1999.

38



Bulletin de Methodologie Soc iologique (59 rue Pouche t, F 750 17 Pari s), October 2002, N. 76 - p. 39

FIG URES

Morepers uas ive

~I'~~===~==~~---Less

persuasive

Figure 1 - Mean Cooperation Rate According to Attitude TowardPersuasion

IIdisag ree

c agree

20

o

40

60

100 ---------.-----

80

It is best to seek animmediate cooperation

It is best to haw the sameintervewe r call back

Figure 2 - Mean Cooperation Rate with Respect to the Strategyfavoured with Respondents in General and with refusers

100 - -----------

80 69,8

60

40

20

o

• yes

Dno

19th. quest. sponsors' rep.

Figure 3 - Mean Cooperation Rate According to PerceivedDeterminants of Survey Cooperation

39



p.40 - Bulletin de h-Uthodo logieSociologique (59 ru t: Poucher. F 750 17 Paris), October 2002 , N. 76

100 ---------

80 ---- - ---- ---

60

40

20

olevel of stress

• often/sometimes

o rarely/never

Figure 4. Mean cooperation rate according to self-reported tailoringaccording to the respondent' perceived stress

':r-:- - ~ ~- ::~J40 - -_~_----

20 - - -----

oLess

autho rttati-..e

Moreautboritatise

Figure 5 • Mean Cooperation Rate According to self-reportedInterviewing Style

40



Bulletin de III/thodologie So ciologique (59 rue Poucher. F 750t7 Paris), October 2002, N. 76 - p.41

API'ENDICES

A endlx A · Factor Anal ses SolutionsPattern matrix

Extraction Method: Alpha. Rotation : Obllmin with Kaiser normalization.N=40

KMO=O.664Explained variance=56.5%

~OCAl - Importance 01 intervi ewer' s vocal characteristics (\1=08624)

Maintaininga conversationaltone .86~ .0871 .023 . 07~

Havinga pleasant voice .828 .023 .03~ .13=\

Clarrtyof pronunciation .804 .033 .042 .064

~9 ease withwhich the interviewercommunicates .67~ .169 .125 .1611

Standardization of pronuncia tion (no accent):" .668 .336 .osa .049

l::ipeaking in a relaxed manner" .66~ .226 .209 . 32~

POSPERS· Favourable atlltude toward persuasion. (\1=0.7472)

""l1itude toward persuasion of reluctant responden ts(INV) I .14<1 ,747. .192 .044

ppinion on the possibility of accepting a refusal .OO~ .639 .003 .283

ppimon on the possibili ty of emphasiZing the voluntary nature of participation .259 .609 .197. .049

~st strategy to adopt with reluctant respondents: insist or call back I .od .532 .047 .016

.019 .160 .861 .006-

.085 .120 .662 .056

.208 .098 .017 .7011

. 1 6~ .192 .173 .571;

.111:1 .258 .059 .474

.426 .482 .155 .031'

~ttitude toward telephon e solicitati on

~tlitUde toward the challenge of convincing respondents

~espect of respondent 's privacy"

ore information on the destination of the call

EGPERS • Negallve altitude towa_r~d-'p_ce_r_cs_cua~s-i-o-n-. c.,(\I_=_0_._6.1_9_6...:)'--- ---,_---,

1\.tlitude toward working in the survey field (INV)

~ttftude toward persuasion of tiesnern respondents( 1)'=0,7685; i f removed) (INV) "

,GRASP ' Belter grasp 01 Interviewi ng task . (\1=0.7._3_6_1):.-__

, ore time to familiarize oneself with the questionnaire

" This item was no t pa rt of the Factor Analys is so lutio n. It was added to produce a scaleidentical to the one used in Lehtone n (1996) . The fidelity o f the scale is low er ed by theinclusion of this var iable ; ho wever, it rem ains at an acceptable lev e l (decrease d from 0.7685 to0.7472)."" In the case of complex va riables, the highest load ing w as the deciding fa ctor. Th e choice ofthis criteria was later conf inned, in all th ree cases, by fide lity analysis .

41



p .42 - Bul/etill tie IlUlhut lo(ogie Sodll(lIgiqu~ (59 rue Poucher . F 750 17 Pari s) , October 2002 , N. 76

---::------,--------- - - - - - --Pattern matrix

Extraction method: Alpha . Rotat ion : Oblimin with Kaiser normalization.N=44

KMO=O.621Explained variance:46.1%

2

.603

.753

.00

. oo~

ffionoN. Importance 01 emotional lactors during survey Introduction phase. ('I1=O._7_6_54-'):.,-__~-___i

ntarviewer's humour -

,Interviewer'S ability to control stress

I'nterviewer's self-confid~nceI ----'-'-----Interviewer's degree of motiltation _

~PMORAL • Importance 01 moral support. ('11 =0.6201)

f'1oral support of superviso r

,Moral support of colleagues

,-IIII,

Pattern matr ix

Extraction method: Alph a. Rotation : Oblimin with Kaiser normalization.

N=44

KMO=O.750

Expl ained var iance=53.3%

2

.488 .095,

.981' .129

------- -.839 .0 1':I

- - - - _ .,BENEFICE . Appeal to respondents ' interest and benefits . ('11 =0.767 4)

e;;;;tnbutes to the impr~vement of things - - ------- - - ---- - - -f---------- -----~ .----------Means of expression on social ~ SS~~ _ (

Respondents personally selected ~

tte~sting questionnaire ('1=0.7858; if re moved fro m faclor)

Interviewer's .comp~~~_ _

Survey topic

.132 .963

~- - - - ---- - --- - -. . 07~ .542

42



Bulletin tie Methodologie Socio logique (59 rue Pouchet , F 750 17 Paris) , October 2002 , N. 76 - p.43

Appendix B - Relationships of motivation, self-efficacy andcharacteristics with interv iewers' cooperation rate

Tests Values

Pearson r Kruska llWall is H

MOTIVATION

Favourable attitudes toward persuasion (POSPERS)

Negative attitudes toward pers uasion (NEGPERS)

Comfort with persuasion

Importance of getting quick consent to participateImportance of having the same interviewer call back in case of a refusa l

SELF·EFFICACYBehaviour

Tailoring the Survey Introduction

According to R's ageAccording to R's education level

According to R's mother tongue or accentAccording to R's stress level

According to R's mood

Arguments used to elicit cooperation

Beneficial to the respondent (BENEFICE)

Nothing to sell

Every opinion is valuableVery shortMean of helping the interviewer

Interviewing Style

Authoritative interviewing style

Friendly interviewing style

Attitudes

Perceived Determinants of Survey Cooperation

Initial impression given to the respondent (INITIAL)

Interviewer's acting skil lsVariation in toneEmotional state of interviewer (PSY)

Vocal characteristics (VOCAL)

Interest for the questionnaire topicKnowledge of the questionnaire topicCall circumstances

Moral support (SUPMORAL)

Field supervisionQuestionnaire quality

Questionnaire length

0 .310 t0 .220

43

0.5213.17 4 •

5.5 16 •



p. 44 - Bulleti n lie /JUthodo logil! Sociologlque (59 rue Po ucher, F 750 ) 7 Paris), October 2002 , N . 76

Tests Values

Pearson r KruskallWa ilisH

Sponsors' reputation

Polling firm's reputation

CHARACTERISTICS

Demographics

AgeSexMother tongue

Language spoken at homeEnglish proficiencyEducation level

Experience and Seniority

Number of work hours

Number of months with current employer

Experience as a telemarketing agentExperience as a customer service representative

Number of polling firms

Number of months of interviewing experience

• p<0.05t p<O.10-NA

44

2.756 t0.685

0.0450.6351.0961.8760.0051.565

0.324 t0.292 t

0.0671.876

-0.1180.000



The Effect Of Interviewer Attitude on Survey Cooperation

Documents

Transcript of The Effect Of Interviewer Attitude on Survey Cooperation