Measuring Group Efficacy in Virtual Teams: New Questions in an Old Debate

10.1177/1046496405284219Small Group ResearchHardin et al. / Measuring Group Efficacy in V irtual Teams

Measuring Group Efficacyin Virtual TeamsNew Questions in an Old Debate

Andrew M. HardinCollege of William and Mary

Mark A. FullerJoseph S. ValacichWashington State University

Group efficacy has received renewed attention in the group literature.Although the relationship between group efficacy and group performance iswell established, debate continues on how best to measure the construct.Although most research has explored this issue using collocated groups, thisarticle examines the measurement of group efficacy in virtual teams andexplores why some measurement methods may be more appropriate for use innoncollocated groups. Fifty-three senior-level university students involved invirtual team projects were administered questionnaires over the course of theirproject. As theorized, data analyses revealed that group efficacy beliefsreached by consensus were significantly higher than those measured by sur-veys administered to individual virtual team members. In addition, support forthe greater predictability of an aggregated method over a group consensusmethod was demonstrated. Follow-up analyses show that group outcome per-ceptions were significantly related to team performance. Implications andfuture research are discussed.

Keywords: group efficacy; virtual team efficacy; virtual teams; performance

Researchers continue to investigate the relationship between group effi-cacy and group performance (Baker, 2001; Pescosolido, 2001;

Salanova, LLorens, Cifre, Martinez, & Schaufeli, 2003; Sargent & Sue-Chan, 2001). As part of this research stream, a debate regarding the measure-ment of group efficacy has developed (Gibson, Randel, & Earley, 2000; Jung

65

Small Group ResearchVolume 37 Number 1February 2006 65-85

© 2006 Sage Publications10.1177/1046496405284219

http://sgr.sagepub.comhosted at

http://online.sagepub.com

Authors’ Note: Please address all correspondence to Andrew M. Hardin, School of Business,College of William and Mary, P. O., Box 8795, Williamsburg, VA 23187-8795; [email protected].

& Sosik, 2003; Whiteoak, Chalip, & Hort, 2004), and several alternativemethods for measuring the construct have been proposed. These methodsinclude independently evaluating each individual’s group efficacy beliefs(Zellars, Hochwarter, Perrewe, Miles, & Kiewitz, 2001), aggregating theindividual self-efficacy beliefs of all group members (Whiteoak et al., 2004),aggregating the individual group efficacy beliefs of all group members (Jung& Sosik, 2003), and finally, allowing the group to assess its efficacy throughdiscussion and group consensus (Gibson et al., 2000).

The debate on how best to measure group efficacy often focuses on whichmethod is best for a particular situation. For example, measuring individualgroup efficacy beliefs is generally well suited for predicting perceptual out-come measures, whereas employing aggregated individual self-efficacybeliefs is useful for measuring performance in situations of limited task inter-dependence. Alternatively, both aggregated group efficacy beliefs and groupefficacy beliefs reached through consensus are generally useful for predict-ing both group outcome perceptions and more objective measures of groupperformance for tasks with high interdependence.

Although the debate about the measurement of group efficacy has gener-ally been addressed using collocated groups, this article extends this debateto the context of virtual teams, where different techniques of measurementmay be more appropriate. This research addresses the following researchquestion: How should group efficacy be measured within virtual teams? Wefirst discuss four prominent methods of measuring group efficacy. Next, weexamine past literature that has compared various methods of measurement.We then discuss how virtual team characteristics may influence the preferredmethod of measurement and then empirically examine two methods of mea-suring group efficacy in the context of virtual teams. Finally, we test the rela-tive predictive ability of a general measure of group efficacy in comparisonwith a more specific measure of virtual team efficacy on outcome percep-tions. We conclude with a discussion of the implications and limitations ofthis research.

Group Efficacy Definition

Before discussing the four methods of measuring group efficacy, we firstneed to clearly define this construct. Group efficacy has been described as anextension of Bandura’s (1997) concept of self-efficacy to groups and thus hasbeen specifically defined as “a group’s sense of its capacity to complete atask successfully or to reach its objectives” (Whiteoak et al., 2004, p. 158).Some authors (Jung & Sosik, 2003) have also used the term group efficacy to

66 Small Group Research

refer simultaneously to both group potency—a measure of a group’s per-ceived general effectiveness (Guzzo, Yost, Campbell, & Shea, 1993) and col-lective efficacy, a measure of a group’s perceived conviction that it can suc-cessfully complete a specific task (Bandura, 1997). Past research focused onthe most appropriate way to measure group efficacy has recognized the con-ceptual overlap of these terms (Jung & Sosik, 2003). For the purpose of thisresearch, we acknowledge this overlap, and refer to both constructs under theumbrella term group efficacy.

Four Methods for Measuring Group Efficacy

In this section we introduce four prominent methods used to measuregroup efficacy, explaining each in terms of its methodology and its associ-ated advantages and disadvantages.

Individual-Level Assessment of Group Efficacy Beliefs

The first method of measuring group efficacy involves assessing eachindividual’s perception of the efficacy of his or her group but without aggre-gating those individual measures to the group level (Zellars et al., 2001).When examining the relationship of efficacy to performance, individualmeasures are assessed against the performance variable of interest—whichin most studies is perceived performance rather than objective performance.When using this method, researchers are generally interested in the predic-tion of individual-level outcome variables such as satisfaction with the teamor job (Zellars et al., 2001). Such a method has the advantage of simplifyingthe research analysis because no special consideration needs to be given tothe group. However, omitting consideration of the group is also the weaknessof this technique, particularly when examining performance measures thatinvolve group-level outcomes.

Group-Level Aggregation of Individual Self-Efficacy Beliefs

The second method of measuring group efficacy is to examine the self-efficacy beliefs of the group members and then aggregate those self-efficacybeliefs to the group level. This method has been used to predict the perfor-mance of groups completing tasks with limited interdependence. Differentforms of aggregation can be used—for example, summing or averaging theindividual self-efficacy beliefs of group members—and then using this com-posite score to predict team performance. Such a measure makes conceptualsense for tasks with limited interdependence because the ability of each

Hardin et al. / Measuring Group Efficacy in Virtual Teams 67

group member to complete his or her task independently is central to theoverall success of the team. This measurement method, however, althoughcapturing some group-level differences, does not ask the individual to assessthe group’s efficacy but rather his or her own efficacy. As a result, thismethod has seen limited use in group research and is most often used only forassessing the performance of groups completing more independent tasks.Nonetheless, when used in these contexts, aggregated self-efficacy beliefscan be useful in predicting certain types of outcomes (Whiteoak et al., 2004).

Group-Level Aggregation of Individual Group Efficacy Beliefs

The third method of measuring group efficacy involves assessing thegroup-level efficacy beliefs of individuals and then aggregating those beliefsat the group level. This method is similar to the former method involvinggroup-level aggregation of individual self-efficacy beliefs, but in this case,individuals are asked to rate their group’s efficacy, which is then aggregatedfor group-level analysis. More specifically, in this method, individual groupmembers are asked about their perceptions of their team’s ability, and thoseperceptions are then aggregated and used for predicting group-level outcomevariables such as group outcome perceptions and performance. This methodhas frequently been used to predict group-level performance variables and isthe method Bandura (1997) recommends. The strengths of this methodinclude its focus on group efficacy—as opposed to self-efficacy—and itsconsideration of the group in the analysis. One potential drawback of thismethod is that it necessitates the use of some type of test to establish theinterrater agreement of team members to ensure the data is meaningful at thegroup level of analysis (Jung & Sosik, 2003; Whiteoak et al., 2004).

Group-Level Consensus Regarding Group Efficacy Beliefs

The fourth and final method of assessing group efficacy is group consen-sus. In this method, the group’s efficacy is assessed by having group mem-bers discuss their group’s efficacy until they reach consensus. This measure-ment method has been used to predict group-level performance variablessuch as group outcome perceptions or group performance (Gibson et al.,2000; Gist, 1987; Guzzo et al., 1993). One obvious benefit of this method isthat it truly represents the group’s composite belief. In addition, this methodalso avoids the need to calculate interrater reliability (Whiteoak et al., 2004).On the other hand, social influence processes could clearly affect the mea-surement of group efficacy using this method. Although the group may cometo a unanimous decision, individual beliefs may still substantially vary. In


addition, group efficacy evaluations may be inflated because of social per-suasion by dominant members of the group (Bandura, 1997).

Studies Comparing MeasurementMethods of Group Efficacy

Several studies have been specifically designed to compare the variousmethods for measuring group efficacy as previously described (Gibson et al.,2000; Jung & Sosik, 2003; Whiteoak et al., 2004). In one such study, Gibsonet al. (2000) evaluated multiple methods for measuring group efficacy in aneffort to empirically establish which measure was most predictive of groupperformance. During that study, Gibson and her colleagues found thatalthough both the group-level aggregation of group efficacy beliefs and agroup-level consensus regarding group efficacy beliefs were positivelyrelated to group outcomes, the group-level consensus regarding group effi-cacy beliefs was most predictive of group outcomes. However, their findingscontradicted the recommendations of Bandura (1997), who cautions againstthe use of group efficacy beliefs reached through group-level consensusbecause of the potential for social persuasion effects.

Interestingly, in a similar study Jung and Sosik (2003) found that groupconsensus regarding group efficacy beliefs was less predictive of group per-formance than was a measure involving group-level aggregation of individ-ual group efficacy beliefs. Although this finding contrasts with the findingsof Gibson et al. (2000), it is consistent with the recommendations of Bandura(1997) as discussed previously. In addition, in agreement with Bandura, Jungand Sosik (2003) found that the statistical means associated with the group-level consensus regarding group efficacy beliefs were significantly greaterthan the statistical means associated with the group-level aggregation of indi-vidual group efficacy beliefs. This may indicate that social influence occur-ring during group discussion is playing a significant role in forming groupefficacy beliefs, possibly inflating the resultant levels of this construct.

Finally, a recent study by Whiteoak et al. (2004) compared three measuresof group efficacy: the group-level aggregation of self-efficacy beliefs, thegroup-level aggregation of group efficacy beliefs, and a group-level consen-sus regarding group efficacy beliefs. During that study, Whiteoak et al.(2004) found that all three measures were similar in their ability to predictgroup goal setting. However, one potential limitation of the study was the useof a task with low interdependence. The issue of low task interdependence isproblematic in generalizing these results, given that group-level aggrega-tions of individual-level self-efficacy beliefs are known to have limited pre-


dictive ability for tasks with high interdependence (Bandura, 1997), asdiscussed earlier.

Table 1 summarizes these different measurement methods, and illustratestheir associated advantages and disadvantages.


Table 1Comparison of Group Efficacy Measures

MeasurementMethod Advantages Disadvantages Important Findings

Individual-levelassessment ofgroup efficacybeliefs

Is predictive ofindividual-leveloutcome variablessuch as satisfactionwith the group.

Eliminates the needfor interraterreliability.

Represents a cross-level analysiswhen attemptingto predict group-level outcomevariables such asgroupperformance.

Successfully predictedindividual-level variablessuch as job satisfaction(Zellars, Hockawarter,Perrewe, Miles & Kiewitz,2001).

Group-levelaggregation ofindividualself-efficacybeliefs

Is predictive of groupperformance fortasks with lowinterdependence.

Is generallyrecognized to beless predictive ofgroupperformance fortasks with highinterdependence.

Did not differ from othermethods in terms of taskdifficulty or the magnitudeof the relationship with goalsetting. However, the taskused had lowinterdependence (Whiteoak,Chalip & Hort, 2004).

Group-levelaggregation ofgroup efficacybeliefs

Is predictive ofgroup-leveloutcome variablessuch as groupperformance.

Requires the use ofsome form ofinterraterreliability test.

Successfully predicted groupperformance.Was found tobe superior to groupdiscussion method forpredicting groupperformance.Group meanswere found to be higher forresponses using discussionmethod than aggregationmethod (Jung & Sosik,2003).

Group-levelconsensusregardinggroup efficacybeliefs

Eliminates the needfor interraterreliability.

Potential forresponse scoreinflation becauseof socialpersuasionfactors.

Found to be superior toaggregation method forpredicting groupperformance.Found thattask-specific measures werepredictive of tasks, whereasgeneral measures werepredictive of the generalability of the group (Gibson,Randel & Earley, 2000)

Group Efficacy in Virtual Teams

Applying the concept of group efficacy to noncollocated teams requiresus to establish a definition of virtual teams. We define a virtual team as agroup of people, often culturally diverse, most of whom are not collocated,who work interdependently with a shared purpose across space, time, andorganizational boundaries using technology (Lipnack & Stamps, 2000).Although this definition acknowledges that virtual teams must still functionas a group, it also highlights two important distinctions between virtualteams and traditional teams. Both the lack of collocation and the need to usesophisticated information technology are factors that add complexity to teaminteractions (Lipnack & Stamps, 2000) and consequently may affect theefficacy beliefs related to those interactions.

Measuring Group Efficacy in Virtual Teams

In addition to the unique nature of a virtual team, which involves the useof technology to overcome a lack of proximity, researchers must also con-sider the complexities associated with measuring the efficacy of this specialform of group (Aubert & Kelsey, 2003). For example, in cases where virtualteam members are unable to meet face-to-face, having group members reacha group-level consensus regarding group efficacy beliefs in a face-to-facediscussion is not possible. Instead, any group discussion by the virtual teamwould require the use of some form of communication technology. However,in such a situation the researcher would now need to consider the effect thatthe particular communication technology used may have on the group dis-cussion process. Specifically, would the communication technology changethe process of social influence, possibly resulting in findings that differ fromthose of face-to-face teams?

In contrast, the measurement of group efficacy using the group-levelaggregation of individual team members’ group efficacy beliefs has beenproposed as a more suitable method for use on virtual teams (Gibson et al.,2000) because of its reliance on data collected from individually adminis-tered surveys. Remote team members can be individually asked to completean efficacy questionnaire, and the responses can then be aggregated.

Considering the unique nature of virtual teams and building on theresearch above, we reiterate our general research question: How shouldgroup efficacy be measured within virtual teams? Similar to past researchthat has compared different measurement methods in face-to-face groups,we compare two popular measurement methods in an examination of virtualteam efficacy: group-level aggregation of individual group efficacy beliefs


and group-level consensus regarding group efficacy beliefs. Group-levelaggregation of individual self-efficacy beliefs was not evaluated in this study,given the interdependent nature of the task. In addition, individual-levelassessment of group efficacy beliefs was also omitted because our purposewas to examine measures of efficacy that focus on the group.

When making predictions about the appropriateness of various groupefficacy measurement methods within virtual teams, past research has indi-cated that the use of communication technology may affect the group’s inter-action (Nunamaker, Dennis, Valacich, Vogel, & George, 1991). Includedamong these effects is the ability to communicate in parallel (Nunamakeret al., 1991) and a reduction in both evaluation apprehension and confor-mance pressure because of anonymity (Connolly, Jessup, & Valacich, 1990).When considering the group interactions necessary to achieve consensusregarding their efficacy, communication technology, consistent with theresearch above, may reduce the ability of one member of the group to domi-nate the group’s discussion because team members can communicate in par-allel. Furthermore, if such discussions are anonymous, evaluation apprehen-sion and conformance pressure should also be reduced, resulting in anenvironment where all team members feel free to contribute (Nunamakeret al., 1991), thus creating a sense of equality among group members. In bothinstances, domination effects should be reduced and the predicted inflationof efficacy evaluations should be less severe than with face-to-face groups.

Other research, however, has shown that more mature teams—those witha shared history—may counteract the influence of communication technol-ogy on team member equalization (Benbasat & L., 1993). Such was thenature of the group interactions in this research, with teams formed at thebeginning of the semester and continuing to interact during several months.In addition, in this research, communication among the various team mem-bers—although private within their respective groups—was not anonymous.Both of these factors should have reduced the influence of communicationtechnology during any discussion directed at reaching a group-level consen-sus regarding group efficacy beliefs. Consistent with the prior literature thatposits that group efficacy beliefs reached through consensus are inflated, wehypothesize:

H1a: Group potency reached through group consensus will be significantlygreater than group potency reached by aggregation.

H1b: Virtual team efficacy reached through group consensus will be significantlygreater than virtual team efficacy reached by aggregation.


Group Efficacy Measurement Methods and Predictability

The aggregation of group members’ appraisals of their group’s capabilityis representative of the coordinative and interactive influences that operatewithin a group and has been shown to be predictive of group outcomes(Bandura, 1997). Conversely, using discussion to reach a group efficacy con-sensus reportedly suffers from the vagaries associated with social persuasionand pressures for conformity as outlined above. In addition, prior empiricalresearch has established a reduction in the predictability of group outcomeswhen using the group consensus method for measuring group efficacy (Jung& Sosik, 2003). Therefore, we propose the following:

H2a: Group potency reached through aggregation will be more strongly related tooutcome perceptions than will group potency reached through consensus.

H2b: Virtual team efficacy reached through aggregation will be more stronglyrelated to outcome perceptions than will virtual team efficacy reached throughconsensus.

Domain-Specific Group Efficacy and Performance

Efficacy is typically considered to be a domain-specific construct, mean-ing that the greatest influence on performance will be caused by the efficacybeliefs that are most closely related to the specific behavior in question(Bandura, 1997). As an example, although a general form of efficacy (e.g.,overall academic performance efficacy) may be somewhat predictive of per-formance related to mathematics, efficacy targeted at one’s ability to domathematics itself should be more predictive of performance. This exampleis similar to the current context, where virtual team efficacy (as a more spe-cific form of group efficacy) should prove to be more predictive than its moregeneral counterpart—group potency. Supporting this assumption, Gibsonet al. (2000) found that general-level efficacy beliefs in the form of grouppotency (defined as the general belief in the team’s effectiveness) were lesspredictive of specific group performance metrics than were more domain-specific group efficacy beliefs. We therefore posit,

H3: Virtual team efficacy reached through group-level aggregation will be morestrongly related to outcome perceptions than group potency reached throughaggregation.


Method

Seventeen teams comprising 53 senior-level management informationsystems students enrolled in a large Northwestern university participated inthe study. The average age of the participants was 26, and approximately70% were male. Virtual teams were formed by randomly combining on-campus and distance-learning students participating in a senior-level infor-mation technology project management course. The on-campus and distancelearning students were unable to meet face-to-face at any time during thesemester. Of the 17 teams, 15 successfully completed both the aggregatedand consensus group efficacy measures and were thus included in theanalysis.

Measures

Group potency was measured using the eight items developed by Guzzoet al. (1993) because of the measure’s consistent use by group efficacyresearchers in studies designed to compare group efficacy measurementmethods (Gibson et al., 2000; Jung & Sosik, 2003). The group potency mea-sure by Guzzo et al. has previously been found to be predictive of groupeffectiveness whether measured through group-level consensus regardinggroup efficacy beliefs or group-level aggregation of group efficacy beliefs(Gibson et al., 2000; Jung & Sosik, 2003). Cronbach’s alpha for the grouppotency measure in the current study was found to be acceptable at .94.

Virtual team efficacy was measured using an instrument developed by theauthors as part of a separate project designed to investigate group efficacywithin the context of virtual teams. Eight items were generated based on areview of the virtual team and group efficacy literature. During item genera-tion, special attention was given to the lack of collocation and use of commu-nication technology by the virtual team members. For example, Item 1 wasworded as “I believe my group has the ability to use communications soft-ware to collaborate with remote group members.”1 The scale was consistentwith the recommendations of Bandura (2001) in that the first portion of thequestion required a yes or no response as to whether the participant believedhis or her group was capable of performing a particular function, whereas thesecond portion asked them to indicate on a scale ranging from 1 to 10 howconfident they were in the team’s ability to perform that function. During theinstrument development process, the virtual team efficacy measure was sub-jected to established methods for instrument development. An exploratoryfactor analysis, confirmatory factory analysis, and an initial model test wereused to establish the convergent, discriminant, and nomological validity


(Chin, Gopal, & Salisbury, 1997) of the virtual team efficacy measure.2

Cronbach’s alpha was found to be .94 for the virtual team efficacy measure inthe current study.

Group outcome perceptions were measured by items forming a compositemeasure of satisfaction with the group, satisfaction with the team deliver-ables, and perceptions of quality regarding the teams’ deliverables. Cron-bach’s alpha for the perceived performance measure was found to be .97.

Team performance was measured by calculating the virtual teams’projectgrades based on the average of the project deliverable scores.

Level of Analysis

The data in this study were analyzed at the group level. Although thegroup-level consensus regarding group efficacy beliefs measurementmethod does not require a test for interrater agreement, the group-levelaggregation of group efficacy beliefs method does (Jung & Sosik, 2003;Whiteoak et al., 2004). Therefore, consistent with previous studies of groupefficacy, a within-group interrater reliability test was performed to ensurethat the group-level aggregation of group efficacy beliefs represented agroup-level effect. In this case, the within-group interrater reliability equa-tion rwg(j) developed by James, Demaree, and Wolf (1984) was used to cal-culate agreement for the aggregated group potency and virtual team efficacymeasures. The average intergroup agreement for the group potency and vir-tual team efficacy measures using the rwg(j) equation was found to be .93and .94, respectively. In addition, the average rwg(j) was calculated to be .93for the group outcome perceptions measure.

Procedures

WebCT, a Web-based learning environment, was used to facilitate thecourse. Student teams were provided with their own private group discussionarea; however, the use of the WebCT group discussion area was not manda-tory and many teams used other types of technology to communicatethroughout the semester. Course deliverables were based on information sys-tems project management and were due throughout the semester. Final teamgrades were based on the average of the deliverable scores.

Survey 1 (Group-Level Aggregation of Group Efficacy Beliefs)

Team member perceptions of the group’s ability were collected after thefirst project deliverable had been completed and feedback was given. Thiswas done to ensure that the virtual team members would have sufficient


information to assess the team’s collective abilities (Jung & Sosik, 2003). Inthe first survey, each student filled out the electronic survey, and theresponses of the individual team members were then aggregated. Both thegroup potency and virtual team efficacy measures were included in thesurvey.

Survey 2 (Group-Level Consensus RegardingGroup Efficacy Beliefs)

For the second survey, team members were asked jointly to fill out an elec-tronic survey used to measure both group potency and virtual team efficacy.The second survey was administered the week following the administrationof the first survey. No deliverables were submitted or graded between sur-veys. The second survey consisted of the group potency and virtual team effi-cacy measures as well as additional measures not included in the first survey.The additional measures were included in an effort to reduce the potential fortesting effects (Shadish, Cook, & Campbell, 2002) and were not used for dataanalysis purposes in the current study. Team members were asked to reach aconsensus as a virtual group. No other instructions were given other than torequest that the students provide documentation as to how they arrived at agroup-level consensus regarding group efficacy beliefs when completing thesurvey. It was stressed that this component of the assignment was as impor-tant as filling out the survey itself and, furthermore, that providing the docu-mentation of the process the group followed was necessary for receiving theresearch participation points.

Survey 3 (Individual Performance Responses)

A final survey, administered before the end of the semester, was designedto measure group outcome perceptions. Group outcome perceptions weremeasured by a composite measure of outcome quality, outcome satisfaction,and group satisfaction.

Results

The members of the teams individually completed Survey 1. The individ-ual responses were then aggregated and compared statistically with thegroup efficacy beliefs reached through consensus.

Responses to Survey 2 were based on the team’s consensus regardinggroup efficacy. To reach a group-level consensus, 9 of the 15 teams chose to


use the asynchronous group discussion board within WebCT, whereas theother 6 chose to use synchronous chat technology. For the 9 teams that choseto use the WebCT discussion area, a log of their discussion was automaticallyrecorded. The 6 teams that used chat technology provided a text file of theirgroup’s discussion to meet the submission requirements for the assignment.

Examination of the discussion and chat logs revealed that all 8 of theteams using the WebCT discussion board averaged the team memberresponses. This was done by having members individually post theirresponses, having one member average those responses and then using theaverage scores to complete the electronic survey. Two of the teams using thesynchronous chat technology reached a consensus through a combination ofboth discussion and averaging—that is, discussion was used to answer mostquestions; however, when disagreement occurred, an average was taken. Theremaining 4 teams discussed the survey items using chat and came to amutual agreement in terms of a group response. One student was generallyassigned the task of recording the group-level consensus regarding groupefficacy beliefs and then completing the electronic survey either as the dis-cussion was taking place or once the discussion had been completed. Table 2depicts these results.

Hypotheses 1a and 1b

Hypothesis 1a was designed to test for potential differences between thegroup-level aggregation of group efficacy beliefs (Survey 1) and the teams’group-level consensus regarding group efficacy beliefs (Survey 2) for thegroup potency measure. To test for differences, a paired sample t test wasconducted. Results revealed that the group-level consensus regarding groupefficacy beliefs associated with the group potency measure (M = 90.92, SD =8.1) was significantly greater than the group-level aggregation of grouppotency (M = 82.41, SD = 8.5, t[14] = 3.48, p = .004).

Hypothesis 1b was designed to test for potential differences between thegroup-level aggregation of group efficacy beliefs (Survey 1) and the teams’group-level consensus regarding group efficacy beliefs (Survey 2) for thevirtual team efficacy measure. The group-level consensus regarding virtualteam efficacy (M = 98.17, SD = 3.4) was found to be significantly greaterthan the group-level aggregation of group efficacy beliefs for the virtual teamefficacy measure (M = 88.36, SD = 6.5, t[14] = 6.05, p = .000). Therefore,Hypothesis 1a and 1b are supported.


Hypotheses 2a and 2b

Hypothesis 2a was designed to investigate whether the group-level aggre-gation of group efficacy beliefs for the group potency measure was morestrongly related to group outcome perceptions than was the group potencymeasure reached through consensus. To test this relationship, two correlationanalyses were conducted. Results revealed that the Pearson correlation coef-ficient associated with the relationship between the group-level aggregationof group potency and group outcome perceptions (.368) was larger than thePearson correlation coefficient associated with the relationship between the


Table 2Methods Used to Reach Collective Efficacy Consensus

Group# Chat WebCT Discussion Average Comments

1 1 1 Individually posted responses, then took average2 1 1 Reached consensus and filled out survey after

chat session3 1 1 Individually posted responses, then took average4 1 1 Individually posted responses, then took average5 1 1 1 Discussed and averaged while chatting (filled

out online survey while chatting)6 1 1 Individually posted responses, then took average

(created Excel spreadsheet)8 1 1 Individually posted responses, then took average

(created Word document)9 1 1 Individually posted responses, then took average

10 1 1 Individually posted responses, then took average12 1 1 Individually posted responses, then took average

(some asynchronous discussion of differencesbut decided to stick with averages)

13 1 1 1 Discussed and averaged while chatting (filledout online survey while chatting)

14 1 1 Individually posted responses, then took average(one member needed to approve averages[self-selected team leader])

15 1 1 Reached consensus and filled out survey afterchat session (largely agreed on everythingexcept one question in the beginning: mostdiscussion off topic)

16 1 1 Reached consensus and filled out survey afterchat session (pretty much agreed some worryabout one question)

17 1 1 Reached consensus and filled out survey afterchat session

Totals 6 9 6 11

group potency measure reached through consensus and group outcome per-ceptions (.228). However, neither measure’s relationship with group out-come perceptions was significant r(13) = .368, p = .177 and r(13) = .228, p =.414, respectively). Because of the findings of nonsignificance, the resultsfrom the correlation analyses provide only limited support for Hypothesis 2a.

Hypothesis 2b was designed to investigate whether the group-level aggre-gation of group efficacy beliefs for the virtual team efficacy measure wasmore strongly related to group outcome perceptions than was the virtualteam efficacy measure reached through consensus. To test this relationship, acorrelation analysis was conducted. Results revealed that the Pearson corre-lation coefficient associated with the relationship between the group-levelaggregation of virtual team efficacy and group outcome perceptions (.505)was larger than the Pearson correlation coefficient associated with the rela-tionship between the virtual team efficacy measure reached through consen-sus and group outcome perceptions (.186). In addition, the aggregated virtualteam efficacy measure’s relationship with group outcome perceptions wassignificant, r(13) = .505, p < .10,3 whereas the virtual team efficacy measurereached through consensus was not, r(13) = .186, p = .506. The finding of asignificant relationship between the aggregated virtual team efficacyresponses and group outcome perceptions, combined with the finding ofnonsignificance for the relationship between virtual team efficacy reachedthrough consensus and group outcome perceptions, provides support forHypothesis 2b.

Hypothesis 3

Hypothesis 3 was designed to test whether the domain-specific virtualteam efficacy measure reached through aggregation was more stronglyrelated to outcome perceptions than the more general group potency measurereached through aggregation. To test this hypothesis, the respective Pearsoncorrelation coefficients associated with the aggregated virtual team efficacyand group potency measures’ relationships with outcome perceptions werecompared. As discussed previously, only the virtual team efficacy measure’srelationship with outcome perceptions was significant, providing support forhypothesis 3. Table 3 depicts the results of the various analyses.

Additional Analysis: Group OutcomePerceptions to Team Performance

To test the relationship between group outcome perceptions and teamgrades, an additional correlation analysis was conducted. Results of the anal-


ysis revealed that perceptions of group outcomes were significantly relatedto team grades (r[13] = .508, p < .10), establishing the relationship betweenteam member outcome perceptions and objective performance.

Discussion

This research makes two primary contributions to the group efficacy liter-ature. First, the debate regarding how best to measure group efficacy wasextended to the context of virtual teams. Second, two popular group efficacymeasurement methods were compared within virtual teams. In both cases,this research helps expand our knowledge of the measurement issues thatmay be encountered by group efficacy researchers in the context of virtualteams.


Table 3Comparison of Two Popular Collective

Efficacy Measures Within Virtual Teams

MeasurementMethod Advantages Disadvantages Result

Group-levelaggregation ofgroup efficacybeliefs

Can be easilyobtainedthrough the useof Web-basedsurveys.

Requires the use ofsome form ofinterraterreliability test.

Group potency measure wasnot significantly related togroup outcome perceptionsa

Domain-specific virtual teamefficacy measure was foundto be significantly related togroup outcome perceptions.b

Group-levelconsensusregarding groupefficacy beliefs

Eliminates theneed forinterraterreliability.

Group discussionmust becoordinatedthrough the use ofcommunicationtechnology.

Potential forresponse scoreinflation becauseof socialpersuasion factors.

Group efficacy responses werefound to be significantlygreater than the average ofthe group level aggregatedresponses for both the grouppotency and virtual teamefficacy measures.c

Neither group potency norvirtual team efficacy werefound to be related to groupoutcome perceptions.d

a. r(13) = .368, p = .177.b. r(13) = .505, p < .10.c. Group potency: t(14) = 3.48, p = .004; Virtual team efficacy: t(14) = 6.05, p = .000.d. r(13) = .228, p = .414 and r(13) =.186, p = .506, respectively.

Supporting Hypothesis 1a and 1b, t tests revealed that group-level consen-sus regarding group efficacy beliefs was significantly greater than the group-level aggregation of group efficacy beliefs for both the group potency andvirtual team efficacy measures. Furthermore, findings revealed that regard-less of whether teams used a synchronous chat technology or an asynchron-ous discussion board to reach consensus, group efficacy scores were inflatedover aggregated individual responses. This finding is particularly interestingin that the social influence purported to occur when using a group consensusmeasurement method occurred even in the asynchronous technology envi-ronment where such influence should have been made more difficult. Thisfinding is most likely the result of the groups’ established histories and thefact that postings made using either type of technology were not anonymous.These results indicate that researchers should be cognizant of potentiallyinflated group efficacy consensus responses in computer-mediated environ-ments, regardless of whether a synchronous or asynchronous environment isused.

Providing circumstantial support for Hypothesis 2a was the finding of alarger Pearson correlation coefficient for the aggregated group potency mea-sure than was found for group potency reached through consensus. However,neither method was significantly related to group outcome perceptions. Pro-viding support for Hypothesis 2b was the finding that the aggregated group-level measure of virtual team efficacy was more predictive of outcome per-ceptions than was the consensus measure. This finding supports the argu-ments of Bandura (1997), who recommends the aggregation method formeasuring group efficacy beliefs, and is consistent with the findings of Jungand Sosik (2003) as discussed previously.

Hypothesis 3 was also supported. Correlation analyses showed that thecontext-specific virtual team efficacy measure was significantly related togroup outcome perceptions, whereas the group potency measure was not.This finding is consistent with efficacy theory (Bandura, 1997), and the find-ings of previous group efficacy researchers (Gibson et al., 2000). Finally, anadditional analysis was conducted to test the relationship between group out-come perceptions and actual performance. In the current case, team memberoutcome perceptions were found to be significantly related to performancemeasured as team grades, providing support for the relationship betweenoutcome perceptions and objective team performance.

This study has several limitations. First, given the difficulty in arrangingvirtual team projects, our sample size was limited. Because of the limitedsample, more sophisticated statistical techniques such as structural equationmodeling could not be used to investigate the paths among these constructswithin an interconnected nomological network. In addition, power was obvi-


ously a concern, and even with the use of a higher significance level (p < .10),power was only sufficient (.8036) for detecting large effects (r ≥ .5). Futurestudies using larger samples should be conducted to corroborate thesefindings.

Although this study had the advantage of observing student teams in a nat-ural environment, the lack of control associated with this field study repre-sents a second limitation, particularly in terms of an inability to establish cau-sation. The lack of a proper counterbalance procedure for the administrationof the individual and group consensus measures is also problematic. How-ever, the procedures followed in this study have been previously used forcomparing group efficacy measures (Jung & Sosik, 2003), and the finding ofinflated group consensus efficacy beliefs is consistent with prior research. Inaddition, the project took place during a 15-week semester, and therefore, theelapsed time between the administration of Survey 1 and 2 was sufficient forminimizing testing effects, yet relatively minimal in relation to the length ofthe project. Future research should be conducted in experimental settings toaddress these limitations.

Additional research should also be conducted in an effort to further inves-tigate the relationship between group efficacy and virtual team performance.Although the relationship of group efficacy and performance among collo-cated teams has been established, other than the modest effort reported here,the same relationship has not been established in virtual teams. As a compo-nent of future research, how group efficacy beliefs develop among virtualteam members should be considered. For example, researchers cannot assumethat team members will develop group efficacy beliefs in a technology-mediated environment in the same way they would if they were collocatedand able to interact face-to-face. In addition, the type of technology used byvirtual teams may also influence the development of group efficacy beliefs.Although the response delay associated with an asynchronous form of com-munication technology such as e-mail may have one type of effect, the use ofa synchronous technology such as videoconferencing may have another, asthe varying impact of communication cues associated with different types ofmedia are well established in the literature (Daft, Lengel, & Trevino, 1987;Lee, 1994; Markus, 1994).

The measurement of group efficacy and its relationship with team perfor-mance has been discussed extensively within the group literature. However,despite the significant attention that group efficacy has received, it is still notbeing consistently measured by researchers. This research further expandsthe discussion regarding the measurement of group efficacy to virtual teamsand, in doing so, contributes to the literature in this area. First, we illustratedthat social influence can result in inflated efficacy estimations in virtual


teams environments in a manner similar to face-to-face teams, regardless ofwhether the technology used is asynchronous or synchronous in nature. Sec-ond, we illustrated the need for domain-specific measures of virtual teamefficacy rather than a reliance on more general group potency measures inthis unique context. This research has provided an initial investigation intothe measurement issues that researchers interested in the group efficacy ofvirtual teams may encounter.

Appendix 1Virtual Team Efficacy Measure4

1. I believe my group has the ability to use communications software to collabo-rate with remote group members.

2. I believe my group has the ability to do teamwork in a distributed environmentif we have access to the appropriate technology.

3. I believe my group has the ability to share information using technology withremote group members.

4. I believe my group has the ability to use communications technology to dowork with people who can’t physically get together to meet.

Notes

1. The complete virtual team efficacy instrument can be found in the appendix.2. Because of the focus of this article on the comparison of group efficacy measurement

methods within virtual teams, detailed instrument development procedures for the VTE measureare not presented. Results of the complete instrument development process are available from thecontact author on request.

3. Power was calculated using GPOWER.EXE. Given a sample size of 15, it was necessary toset the significance level at .10 to achieve an acceptable level of power (.8036). Even using a sig-nificance level of .10, power was only sufficient for detecting large effects (r ≥ .5).

4. The authors originally generated eight items but eliminated four during the confirmatoryfactor analysis.

References

Aubert, B. A., & Kelsey, B. L. (2003). Further understanding of trust and performance in virtualteams. Small Group Research, 34, 575-618.

Baker, D. F. (2001). The development of collective efficacy in small task groups. Small GroupResearch, 32, 451-474.

Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W. H. Freeman/TimesBooks/Henry Holt.


Bandura, A. (2001). Guide for constructing self-efficacy scales. In G. V. Caprara (Ed.), Lavalutazione dell ’autoefficacia [The assessment of self-efficacy]. Italy: Erickson Trento.

Benbasat, I., & L., L. (1993). The effects of group, task, context, and technology variables on theusefulness of group support systems: A meta-analysis of experimental studies. Small GroupResearch, 24, 430-462.

Chin, W., Gopal, A., & Salisbury, W. D. (1997). Advancing the theory of adaptive structuration:The development of a scale to measure faithfulness of appropriation. Information SystemsResearch, 8, 342-368.

Connolly, T., Jessup, L., & Valacich, J. (1990). Effects of anonymity and evaluative tone on ideageneration in computer-mediated groups. Management Science, 36, 689-703.

Daft, R. L., Lengel, R. H., & Trevino, L. K. (1987). Message equivocality, media selection, andmanager performance: Implications for information systems. MIS Quarterly, 11, 355-366.

Gibson, C. B., Randel, A. E., & Earley, P. C. (2000). Understanding group efficacy: An empiricaltest of multiple assessment methods. Group & Organization Management, 25, 67-97.

Gist, M. (1987). Self-efficacy: Implications for organizational behavior and human resourcemanagement. Academy of Management Review, 12, 472-485.

Guzzo, R. A., Yost, P. R., Campbell, R. J., & Shea, G. P. (1993). Potency in groups: Articulating aconstruct. British Journal of Social Psychology, 32, 87-106.

James, L. R., Demaree, R. G., & Wolf, G. (1984). Estimating within-group interrater reliabilitywith and without response bias. Journal of Applied Psychology, 69(1), 85-98.

Jung, D. I., & Sosik, J. J. (2003). Group potency and collective efficacy: Examining their predic-tive validity, level of analysis, and effects of performance feedback on future group perfor-mance. Group & Organization Management, 28, 366-391.

Lee, A. S. (1994). Electronic mail as a medium for rich communication: An empirical investiga-tion using hermeneutic interpretation. MIS Quarterly, 18, 143-157.

Lipnack, J., & Stamps, J. (2000). Virtual teams: People working across boundaries with technol-ogy. New York: John Wiley and Sons.

Markus, M. L. (1994). Electronic mail as the medium of managerial choice. Organization Sci-ence, 5, 502-527.

Nunamaker, J. R., Dennis, A. R., Valacich, J. S., Vogel, D. R., & George, J. R. (1991). Electronicmeeting systems to support group work. Communications of the ACM, 34, 40-61.

Pescosolido, A. T. (2001). Informal leaders and the development of group efficacy. Small GroupResearch, 32, 74-93.

Salanova, M., LLorens, S., Cifre, E., Martinez, I. M., & Schaufeli, W. B. (2003). Perceived col-lective efficacy, subjective well-being and task performance among electronic work groups:An experimental study. Small Group Research, 34, 43-74.

Sargent, L. D., & Sue-Chan, C. (2001). Does diversity affect group efficacy? The interveningrole of cohesion and task interdependence. Small Group Research, 32, 426-451.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-ExperimentalDesigns for Generalized Causal Inference. Boston: Houghton Mifflin.

Whiteoak, J. W., Chalip, L., & Hort, L. K. (2004). Assessing group efficacy: Comparing threemethods of measurement. Small Group Research, 35, 158-173.

Zellars, K. L., Hochwarter, W. A., Perrewe, P. L., Miles, A. K., & Kiewitz, C. (2001). Beyondself-efficacy: Interactive effects of role conflict and perceived collective efficacy. Journal ofManagerial Issues, 13, 483-499.


Andrew M. Hardin is an assistant professor of information technology and operations manage-ment at College of William and Mary. He received his Ph.D. in management information systemsfrom Washington State University in 2005. His research interests include virtual teamwork, tech-nology mediated learning, research methodologies, and the information systems workforce. Hisresearch has appeared in journals such as Small Group Research and Group Decision and Negoti-ation. Prior to undertaking his doctorate at Washington State University, he worked for severallarge corporations including Texaco Refining and Marketing where he was employed as bothplanner and material coordinator.

Mark A. Fuller is an associate professor and chair of the Department of Information Systems atWashington State University. He received his Ph.D. in management information systems fromthe University of Arizona in 1993. His research interests include virtual teamwork, technologymediated learning, trust, and e-commerce. Past research has appeared in journals such as Infor-mation Systems Research, Journal of Management Information Systems, Journal of Organiza-tional Behavior, Group Decision and Negotiation, Decision Support Systems, and the Journal ofInformation Systems Education.

Joseph S. Valacichis is Hubman Distinguished Professor in the Information SystemsDepartment at Washington State University. His primary research interests include technology-mediated collaboration, mobile and emerging technologies, e-business, human-computer inter-action, and distance education. He has published more that 50 referred articles in journals such asMIS Quarterly, Information Systems Research, Management Science, Academy of ManagementJournal, Communications of the ACM, Decision Science, Organizational Behavior and HumanDecision Processes, Journal of Applied Psychology, and Journal of Management InformationSystem.


Measuring Group Efficacy in Virtual Teams: New Questions in an Old Debate

Documents

Transcript of Measuring Group Efficacy in Virtual Teams: New Questions in an Old Debate