An assessment of group support systems research: results

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

An Assessment of Group Support Systems Research: Results

Jerry FjermestadSchool of Management

New Jersey Center for Multimedia ResearchNew Jersey Institute of Technolog

Newark, New Jersey [email protected]

Starr Roxanne HiltzDepartment of Computer and Information Science

New Jersey Center for Multimedia ResearchNew Jersey Institute of Technolog

Newark, New Jersey [email protected]

heenur

thhaslleSe

thr chedoreeavolioS

o

ilt

o

sar

ntlaags

ons,o

daree

e

lesoty),, and

sntalibed78d ad

the

eadeve

ngsg582id-ast toe are

sr

isheSS

AbstractThis paper reports on the results of 200 Group SupporSystems (GSS) experiments that have been publisbetween 1971 and June of 1998. 1582 hypothesresulting from pairings of independent and dependevariables were categorized and analyzed. The resshow that the modal outcome for GSS systems compato face-to-face (FtF) methods is "no difference," while overall percentage of positive effects for hypotheses tcompare GSS to FtF is a disappointing 16.6%. Theresults vary by task type, the number of experimentagroups per cell, and the type of dependent variabmeasured. The purpose of this paper is to aid the Gresearcher by presenting detailed results of what has bstudied and found in previous experiments, along with adiscussion of what needs to be studied.1. Introduction

How can networked computers be used to improve process and outcomes of group meetings? For ovedecade, this has been the Holy Grail sought by researon Group Support Systems (GSS). They have devisvariety of computer based tools and processes suppby software and/or human roles, and used with differcombinations of communication modes; studied theffects for different types of groups and tasks; and hmost typically assessed their success through contrexperiments. Though there have been some prevsummaries and meta-analyses of the results of Gexperiments, none of them have included the full bodyresearch to date.

The first part of this assessment Fjermestad and H[13, 14], presented a theoretical framework whichorganizes all of the variables that have been studiedtheorized to be important for understanding theconditions under which GSS’s are effective. Thiframework categorizes the variables into ContextuIntervening, Adaptation, and Outcome Facto(Fjermestad, [12], gives a full explanation of thetheoretical framework). Contextual (or Independevariables include characteristics of the particutechnology (GSS) being used, of the group, and the tenvironmental and organizational contexts. Interveninvariables refer to meeting characteristics such a

0-7695-0001-3/99 $1

dstltsedete

Sen

eaers atedntieledusSf

z,

l,

)

sk,

experimental design, session length, number of sessiand training. The adaptation (adaptive structuration) interaction process of the group includes such things astheir level of effort, their attitude toward the GSS, anparticipation patterns. They are the variables that controlled by the group on an individual or collectivbasis.

Outcomes are the result of the interplay of thintervening factors and adaptation of the group with thecontextual factors. These results or dependent variabinclude efficiency measures (e.g., calendar time tdecision), effectiveness measures (e.g., decision qualisubjective satisfaction measures, consensus measuresusability of the system.

Distributions for the frequency with which the varioucontextual and intervening variables have been used iGSS research, as well as for aspects of experimenmethodology used in this area of research, were descrin an earlier paper [14], which used the study (1experiments) as the unit of analysis. We also presentelist of experimental studies published to that date, ansummarized the methodology used, including thosevariables that have been manipulated or controlled (contextual and intervening factors).

In this article, we will summarize the findings of theexperiments, in terms of the adaptation and outcomfactors studied, and the generalizations that can be mon the basis of variables for which numerous studies habeen conducted. We analyze and summarize the findiof the body of experimental work on GSS by conductinstatistical analyses on a database that includes 1hypotheses from all published experiments through m1998. The objective is not just to summarize what hbeen studied and learned in the past, but to poinpotentially fruitful research directions in the field for thfuture. What areas have been relatively neglected andin need of further study? What kinds of GSSconfigurations appear to have relatively better impactthan others, and to thus be promising for furthedevelopment and study?

First we describe the methods used to conduct thanalysis. This is followed by a summary of results for tmost frequent experimental design, which compares G

0.00 (c) 1999 IEEE 1

oivy)t fousin

ofruea

o

hahio

mk

t

le

eow

l

tro aw

in

oeo

nt-

en aent,oseofu

on theentstedni-xyesionross- foron, ons

hele1;specte inos, atan of

anr

ts

nd aretedich


to the non-supported FtF mode, in terms of the kindsoutcomes for which GSS’s are more or less effect(Table 1). Then we ask, under what experimentallinduced circumstances is GSS most effective (Table 2The subsequent analyses begin to explore the data thaunderneath Tables 1 and 2 by looking at the detailsresults for sets of independent and control variables which there are a substantial number of studies: groproximity, anonymity, time dispersion, GSS levelcommunication mode, group size, task type, sesslength and number of sessions. Finally, we recommedirections for future research2.0 Method

A bibliography of 230 papers representing 200different experimental studies of group support systemswas prepared (http://eies.njit.edu/~jerry/gss/GSSref.htmIf two papers reported the same design, number subjects and other methodological details, they weconsidered to be the same experiment. To meet ocriteria for inclusion, first of all, the publication had to ba study of groups, which we defined as comprising at lethree members. Simmel [35, 36], was the first tconvincingly make the case that a dyad is fundamentallydifferent than larger groups, because the triad is tsmallest size aggregation in which there can be "majority" which stands against a minority. Secondly, tstudies used a computer-based GDSS (Group DecisSupport System, generally employed in a “decisionroom”) or GCSS (Group Communication Support Systea computer-mediated communication system which linpeople in different places and/or communicating atdifferent times) with at least minimal features designedsupport group communication and decision makingprocesses. Third, the study was actually a controlexperiment: there were two or more conditionsdeliberately created and contrasted; other variables wcontrolled in some manner; and there was at least independent and one dependent variable, which measured and statistically analyzed. Finally, the studhad to be published, in English, in a refereed journamajor conference proceedings; e.g., unpublisheddissertations or conference presentations or book chapare not included. A few studies fail to meet one or moof these criteria, but are cited heavily in the literature, seem to be significant studies, so they are included"non-conforming studies." Undoubtedly, there are a festudies that we missed; and, of course, more are bepublished each month.

All studies were categorized and coded by the authand then put into a data base [17] which uses thexperiment as the unit of analysis, and records and represults at the level of the individual hypothesis within

0-7695-0001-3/99 $1

fe

? lieofrp,ond

l).

st

e

en

;s

o

d

reneas

o

erse

s

g

rs

rts

each experiment, yielding a total of 1582 independedependent variable pairings.2.1 Analysis vs. Meta-Analysis

A meta-analysis works with percent of variancexplained. Unfortunately this statistic is not reported ilarge number of published studies. For this assessmwe strove to include all experimental results, even thwhich employed non-parametric tests. The unit analysis is the hypothesis; results are included in ocounts only if they were significant at the .05 level.3.0 Tests of Hypotheses

What are the results of all these tests of effects dependent variables? In order to try to make sense offindings, we have identified several sets of independvariables that have been explicitly tested or at learecorded quite frequently: the specific type of GSS us(synchronous, asynchronous, etc.), mode of commucation, group size, and session length (which is a profor task complexity). For all studies testing hypotheswhich compared GSS to face-to-face (FtF) communicatmodes, these independent variables have been ctabulated by all dependent variables combined, andseparate groups of dependent variables (satisfacticonsensus, efficiency, effectiveness, all structurationprocess variables, and usability). The following sectiopresent the results of these analyses.3.1 Which Independent Variable Manipulations HaveMost Frequently Resulted in Significant DifferencesBetween GSS Supported and Unsupported Groups?

Table 1 shows the hypothesis counts for all of tindependent variables. The first independent variabcategory examined is technology, following from Table 63% of all the hypotheses have been tested on some aof technology. The results shown in Tables 1 and 2 arterms of the proportion of studies comparing GSS t(unsupported) FtF which resulted in “positive” effect(GSS results were significantly better than FtF resultsthe .05 level), “negative effects” (FtF was better thGSS), or no significant effects; as well as the numberstudies using these variables for which no main measureswere reported, or for which relationships other thcontrasts between FtF and GSS groups were tested (otheeffects and interaction effects). The column labeled“Percent positive effects” is the ratio of significan“positive” findings to the total number of positive plu“negative” plus “no effect” findings. The “positive ratio”column computes only the ratio of positive results tonegative results; 1.0 would be equal, for instance, anumbers greater than 1.0 indicate that positive resultsmore likely than negative results in the body of compleexperiments. In the discussions, in determining wh

0.00 (c) 1999 IEEE 2

e

ateall

tsf

lt i50SSantuio

tats.

e

orsesltery

ult. Ind ws,l

.hasselallndsu is

s.t innl

to

av innd

o ofs.

ces

inles

],mennotrs

tn ofisve in.,

isreto ton ondenre

kre a

t

her 6%ts.

thave% a).dvece


variables have produced the most positive results, wmust take both of these ratios into account.

The overall results, shown in the bottom row, indicthat there are 16.6% positive effects observed for hypotheses tested in all GSS studies. Furthermore,negative effects only slightly out number positive effec(164 vs. 158, respectively), resulting in a ratio opositive/negative effects equal to 1.0. The modal resu“no significant effects” observed, (628 out of the 9hypotheses for which main effects for GSS vs. no Gwere reported). These are fairly discouraging results suggest that a change in focus and/or methods of fuGSS experiments would be advisable (see the Discusssection at the end).

Communication mode represents 33.5% of the tohypotheses and 29.2% of all of the positive effecManipulations in process structuring have been examinedprimarily for the way they interact with technology, raththan for main effects (173 hypotheses).

The other Context Factors and Intervening Factrepresent only 37% of the total hypotheses. Thindependent variable categories have resuoverwhelmingly in “no difference” and accounted for vefew positive effects.3.2 For Which Dependent Variables is GSS MostEffective?

Table 2 examines relative favorableness of resfrom the other direction, dependent variable categoriesgives the overall count by dependent variable (type acategory) for the results of the tests of hypotheses. Ashave noted above, for most dependent variabledisappointingly, “no significant difference” is the modaresult. Negative effects are more prevalent than positiveeffects for many categories of dependent variablesProcess gains were somewhat less likely with GSS twith FtF, as were outcomes categorized as “proceissues” in Table 2, while process losses were just as likto occur with GSS as without. Since GSS are generintended specifically to increase process gains adecrease process losses, this is a most discouraging re

The (important) exception to these dismal resultsthe key dependent variable of effectiveness (with apositive ratio of 1.7), as well as structuration variableHowever, for consensus, GSS had a negative effec15.4% of the tests, compared to positive results for otwo hypotheses, yielding 7.7% positive outcomes foconsensus. It is obvious that the relative lack of abilityreach consensus is a problem for groups using GSS.3.3 Comparisons Among GSS Technologies

The results, in general, show that CMC systems hyielded about the same proportion of positive effectscontrast to GSS systems, (15.8% and 17.6% for CMC a

0-7695-0001-3/99 $10

s

dren

l

ed

st

e

n

lt.

e

GSS, respectively). GSS systems have a higher ratipositive/negative effects, 1.3 to 0.7 for the CMC systemHowever, there appears to be no substantial differenamong the GSS decision room systems, the synchronousCMC systems, and the asynchronous CMC systemsterms of relative outcomes, when all dependent variabare considered.3.4. Comparisons Among Process Structures

3.4.1 Anonymous vs. Identified Conditions:Aspointed out by Valacich et. al. [41] and by McLeod [25anonymity has different components or “types.” In socases, the group knows who the members are, but caidentify individual contributions. In other cases, membedo not even know who belongs to the group. Individualcontributions may be simply “ anonymous,” or noattributable to any individual; or they may use “penames” or “pseudonyms, in which case there is a kinda “personna” to whom a series of contributions attributed. These different forms of anonymity may havery different effects. Overall, there is an advantagefavor of identified versus anonymous conditions; e.g19.1%and 16.7%, respectively. However, anonymityone of those variables for which interaction effects aespecially prominent. For example, drilling deeper inthe data set (not shown), the positive effects increase24% for anonymous groups working on idea generatiotasks and decrease to 13%and 10%, respectivelyintellective and decision making tasks. Conversely, unidentified conditions, the highest positive effects are odecision making tasks (22.6%), while lower levels aobserved on intellective tasks (9.6%) and on ideageneration tasks (18.8%). Thus, the results suggest thatanonymity may be best suited for creativity tasks, fowhich GSS outcomes are relatively better than other tastypes. The intricacies of the effects of anonymity abeyond the scope of this paper, and are worthy ofseparate analysis and summary.

3.4.2 Time Dispersion: Synchronous studies accounfor 91.5% (1447 out of 1582) of all the hypothesesexamined and produced 16.7% positive effects. On tother hand, asynchronous hypotheses only account foof the total and resulted in 20.32% of the positive effecThus, the probability of positive outcomes is slightlbetter for asynchronous technologies.

3.4.3 Proximity: Decision room settings accounfor 64% of the hypotheses that have been tested, and produced 18.2% positive results (compared to 16.6positive results for all proximity modes combined) andpositive ratio of 1.3 (compared to an overall ratio of 1.0Distributed settings account for the other one thir(35.9%) of the hypotheses and yield only 14.1% positiresult. In addition, distributed settings have almost twi

.00 (c) 1999 IEEE 3

fres

o-teoso

o"hemsarorwe

wid

fld

e fo ath 0ec tenegveanhe

pen

ifobel iv

tl

hatry,s,”ntsd

.nde

to

up

are

tss.ct

nynly

nhendts ofhanis10ical

20up

sesoing

nowhltsel

at itare


as many negative effects as positive effects (ratio opositive/negative effects = 0.6). Thus, results arelatively more favorable for decision room conditionthan for various types of distributed (synchronous orasynchronous) conditions.

Obviously, more research is needed in theasynchronous and distributed areas. With the millionspeople using the Internet and the proliferation of Webbased GSS and CMC systems, this is the type of systhat is most likely to be used in the future, whereas mof the studies have been on synchronous decision-roGSS.

3.4.4 Levels:The "level" of the GDSS or CMCsystem is a rough coding of its sophistication in termsGDSS features, and follows the descriptions of "level 1and "level 2" systems by DeSanctis and Gallupe [8]. Tmajority of the systems used (65%) are level 1 systeand another 3.4% compare level 1 and level 2 systemSome of the CMC systems are actually just email and not structured to keep an organized and searchable recof the group discussion, as a conference is, but if that what was used, it is included. However, nine of thstudies use CMC systems so "poverty stricken" that have labeled them "level 0" because they do not proveven the minimal features that would facilitate a richgroup discussion. The most restrictive and limited othese are systems with fixed screens designed with fieto be filled in to support highly structured decisionmaking for a specific task. For instance, if there wer"choice dilemma" task, the screen might have a field the short problem description, and a field for inputtingnumber between 1 and 10, which is the "answer" to choice. At the bottom of the screen of these "levelCMC facilities there is often only a single line, or in somcases, only a half of a line, which is the entire spaallowed to compose and send free text communicationother group members. "Chat" systems with split screthat allow one to view only a few lines of text, providlittle or no editing, and display what everybody is typinas they type it, letter by letter, are also classified as "le0." They give the users no control over what they see what and when they send, nor do they support tcommunication of well thought out and editedcontributions, or reviewable transcript of the groudiscussion. Many of these "level 0" systems also severything anonymously automatically.

The results are somewhat surprising, in that thereabout the same 15% proportion of positive outcomes Level 0 systems as for level 1 systems; this may because the problems for which the simple systems wconstructed are also very simple. However, the levesystems have a much better ratio of positive to negat

0-7695-0001-3/99 $1

f

mtm

f

s,.edas

ee

s

a

e"

eos

ld

d

s

ere1e

outcomes (0.9 vs. 0.5). Level 2 systems have slighmore positive results than less sophisticated systems.3.5 Comparisons Among Group Factors

3.5.1 Group Size: Trying to cross tabulate results bthe size of the groups used is complicated by the fact ta large number of experiments allowed group size to vaeither deliberately or accidentally (because of “no showetc.) The categories which we used are experimewhere the group size varied roughly between three anfive members (small groups); 6-10 members (mediumsized groups), and group sizes of 10-20 (larger groups)

The seminal theoretical framework by DeSanctis aGallupe [8] used “small” vs. “large” groups as one of thprimary contingency factors that must be taken inaccount in designing GSS, and subsequent GSSresearchers appear to concur in the belief that the size ofthe group interacts with the types of communicationtechnology and tools that will be helpful to support groprocesses. However, there is no agreed upon dividing linebetween “small,” “medium” and “large” groups, andstudies of larger groups (e.g., ten or more members) logistically difficult because of the size of the GSSfacilities needed and/or the total number of subjecrequired for an experiment utilizing larger group sizeThis circumstance, in addition to the disappointing fathat a substantial number of studies have not controlledgroup size rigorously enough to fit into even a fairly broadcategory, means that we do not have very mahypotheses that have been tested for larger groups (o25 total).

Nevertheless, there i s a very striking difference iresults reported associated with size of the groups. Tsmaller groups have 10.0% to 16.8% positive effects aas many positive effects for GSS as negative effecreported. For medium sized groups, the proportionpositive effects drops to about 9%, and there are less thalf as many positive effects as negative effects. Thdifference between groups of 3 to 5 vs. groups of 6 to would not be expected on the basis of current theoretframeworks, and needs to be directly investigated.

When we move to the larger sized groups of 10 to members, the predicted increase in utility of GSS as grosize increases does manifest itself. 36.4% of hypotheabout these larger groups resulted in GSS groups dbetter than unsupported groups, and there were negative effects observed. This may be one reason field studies of GSS seem to give more positive resuthan laboratory studies; field studies are much more likto be with larger groups.3.5.2 Subject Type: Turning to the type of subjectstudied, we once again note that the types of groups this logistically easy to study (undergraduate students)

0.00 (c) 1999 IEEE 4

arho

hk th reenre0se

a

chom

nup thtain fo

gveoe

s b

tph

ine

heos

s

C inucS

ve i

ns,

ite:Cs onf

ucho

af [7,usea

ps toe.8

irdeses

ectslessith

ee

SS

s.

up.gthakeSS

the

are forhe an 30%.fo

ve


much more prevalent than the types of subjects who most likely to be able to benefit (graduate students, ware likely to have had some work experience, o‘professionals,” which in this context means anybody whas a job rather than being a student). When we loothe pattern of results, they are just about opposite to pattern of subjects used. For groups composed solelyundergraduate students, only 15.4% of results wepositive, and negative results were as likely as positivresults. By contrast, for studies using graduate stude(44.4% of tests of hypotheses) yielded slightly mopositive results for GSS, and positive results were 6.times as likely as negative results. For groups compoof professionals rather than students, results wereoverwhelmingly “no difference” (73%), with 13.5%positive and 13.5% negative. These latter percentagesbased on only 7.8% of the hypotheses (123 out of 1582);so few that we really cannot say that we know very muabout non-student users of GSS in terms of results frcontrolled experiments.3.6. Comparisons Among Task Types

Both Hollingshead and McGrath [20] and Dennis,Haley, and Vandenberg [7] suggest task type camoderate the effect of a GSS. Whereas all GSS groaggregated together have positive effects in 16.6 % ofhypotheses, when organized by task type, the percenincreases to about 21% for task types 4 (decision makand 5 (mixed motive), whereas it decreases to 13.6%task type 3 (Intellective). Though task type 2, ideageneration, does not have a particularly high percentaof positive effects, it does have relatively few negatiresults, and thus an outstanding ratio of positive tnegative effects (2.4). The best proportion of positiveffects appears for task type 1, planning tasks, but thibased on very few studies and hypotheses and thus cantaken as suggestive of an area for future experimentation,at best.

These results are then broken down accordingwhether they occurred for GSS type support or CMC tysupport. Several unexpected results are observable wlooking at the interaction of task type and system typethis manner. Though the overall proportion of positiveffects is the same for the two types of systems, tpattern of variation around the mean differs. This is mstriking for task type 2: it is obvious that CMC systemare fairly effective when used with tasks requiringdecision making. For all variations (CMC Synch, CMAsynch, CMC Combined; this breakdown is not shownthe table) positive effects are close to 30.0% and are mmore likely than negative effects. By contrast, for GSsystems, positive effects are only 16.6% and negatieffects are more likely. Thus, results suggest that CMC

0-7695-0001-3/99 $10

e

oateof

ts

d

re

segeg)

e

ise

oeen

t

h

s

highly effective when used in decision making situatioas compared to GSS.

For mixed motive tasks, the pattern is the opposGSS systems appear much more effective than CMsystems, though the number of studies and hypothesewhich this contrast is based suggests that direct tests osuch an association need to be carried out. FoIntellective tasks, the differences are in the samedirection, and though not as striking, are based on mmore data; positive results for CMC systems are a l7.9%, compared to 14.4% for GSS systems.

However, the results for GSS groups using idegeneration tasks to do not support the observations o15, 20], who suggest that task type moderates GSS and that GSS groups perform better when using idegeneration tasks. The positive effects percentage dro15.0% for GSS while it is 18.9% for CMC systems. Thratio of positive to negative effect is higher for GSS at 2(vs. 1.4), however

Further analysis, adding group size as a thdimension (not shown) reveals that of the 501 hypothexamined for GSS groups using idea generation tasks,52.9% had six groups or less per cell. The positive effdrop to 9.4% for the studies which had six groups or per cell, (with the vast majority of these experiments wsmall N's of groups per condition showing "no significantdifference"), while for studies with 7 or more groups pcell the positive effects are increased to 29.0%. Thresults reported here support Fjermestad and Hiltz [15]assertions that statistical power is very important in Gresearch. The experiments which had at least 7 groups omore per cell support the observations of [7, 20] that GSSgroups perform better when using idea generation task3.7 Comparisons Among Intervening Factors

Session length can be considered a proxy for theoverall complexity of the task being solved by the groMany experiments reported variations in session lenrather than holding it constant; however, we can mrough categories for the synchronous modes of Gconsisting of 10-30 minutes as the shortest sessions, 30-60 minutes as medium length, and one to two hours aslonger meetings. After this, meeting length goes to days to weeks range, for the asynchronous conditions.

For all dependent variables combined, there inconsistent changes in the proportion of outcomeswhich GSS is significantly better than FtF groups. Thighest positive effect (25.2%) is in the sessions withlength of less than a half-hour. For sessions betweeand 59 minutes the percentage drops to a low of 9.4The positive effects then increase to 19.8% of results sessions of 1-3 hours and 20.3% for asynchronousmeetings. The best overall ratio of positive to negati

.00 (c) 1999 IEEE 5

ion

Sbeg arcui

bessou

nt up idn

ultsin

enupLeses

Singnt

fonaene 2

ted Thompsheticbeten

an c

velpk

fse

e

gorys ofof aesses

are

ofatinthethat(asngisalle

ps

re

hatsebed:is

ndthe

ed

r

e

S

nd

eld


effects occurs when there is no limit placed on sesslength. Though based on a relatively small number ofhypotheses, this suggests that even in experiments, Gshould more frequently be used as it would “naturally” adapted by groups, that is, let the groups take as lonthey need to finish. We know that generally groups less efficient with GSS than FtF; if session length is off at what is needed for FtF groups to complete thetask, then it seems obvious that this will often not enough time for groups which are typing their idearather than speaking them, and that quality will thusuffer because of not enough time to complete the grprocesses.

4. Summary and DiscussionOur objective was to present a detailed assessme

the empirical results for experimental studies of GroSupport Systems. In particular, we are interestedsummarizing the findings on the extent to which anconditions under which GSS is “better” or “worse” thaunsupported face-to-face meetings. Overall, the ressuggest that the research shows an overwhelmtendency to find "no significant differences" betweunsupported face-to-face modes and the types of grosupport systems that have been studied thus far. than one-fifth of the findings, overall, support hypothethat GSS use is better than face-to-face methods (GSFtF). This is consistent for both CMC systems (uscomputer networks to connect participants in differelocations, either synchronously or asynchronously) ordecision room GSS. Adding task type as an additiocontrol variable, we observe more positive results whCMC systems with task type 4, and GSS with task typare compared to f ace-to-face conditions.

Thus, similar to the observations made bHollingshead and McGrath [20] and the results reporby Dennis, et. al. [7], task type moderates GSS use. results reported here suggest that GSS decision rotechnology has the highest probability of aiding grouperforming idea generation tasks in comparison to ottask types, provided that the study had sufficient statispower of at least seven groups per cell. This may because idea generation benefits from independencogitation, and does not require a great deal of agreemor other forms of social-emotional interaction.

CMC systems provide a less “rich” environment thdo decision room GSS. Perhaps, the group membershave time to reflect and digest the decision alternatibefore deciding on a final decision. This might heexplain why CMC groups working on a decision tashave more positive results.

Based on 200 experiments which span 20 years oresearch, we observe 16.6% overall positive effect (

0-7695-0001-3/99 $10

S

as

t

p

of

n

g

ss

>

rl

,

e

ral

t,

ans

e

Table 1) due to GSS use compared to face-to-facmethods. We will set this limit as our benchmark fointerpreting the detailed results on the dependent catevariables. Then, the overall results on the categoriedependent variables (Table 2) suggest that the use GSS has relatively more positive impacts on effectiven(21.4%), process variables (18.8%), role outcom(18.2%), and structuration (25.0%). Negative impacts associated with efficiency (44.8%) which is mostldecision time. GSS groups take longer to complete theitasks than do FtF groups.

In addition to variations in associated with the typeGSS and its interaction with task type, we note thresults are significantly more positive, particularly terms of effectiveness, when GSS is used on larger rathan small groups, and on more complex problems can best benefit from computer support for analysis measured by the proxy for task complexity, total meetitime required). The irony, as noted in the first part of threview, is that most experiments have been on smgroups doing simple tasks, probably because these are theasiest to study in terms of quickly running grouthrough experimental conditions to obtain “results.”

Using these results as the starting point for futuresearch, our objective is to find ways to improve thedesign and use of GSS. Field studies tend to show tthe use of GSS actually reduces meeting time, thuefficiency is improved. One could also summarize thresults in terms of an experimental design that would most likely to generate relatively positive effects; it woul• Use a “level 2” system with sophisticated analystools built in.• Use subjects who are likely to be knowledgeable amotivated about the task; e.g., graduate students rathan undergraduates.• Aggregate the subjects in medium to large sizgroups—at least 6, 10 or more is even better.• Make sure that there are sufficient groups pecondition to provide adequate statistical power.• Give the groups a facilitator and plenty of tim(ideally, unlimited time).• Use a task type that is most likely to benefit from GSand is matched to the communication medium.• A planning task is especially likely to benefit fromGSS.• If you have a decision (preference) task, use CMC,and if an intellective task, use decision room GSS.• Measure structuration as an intervening variable aaspects of effectiveness as dependent variables ; don’tmeasure efficiency (time to completion) or satisfaction. Obviously, this “recipe to produce significant positivresults from GSS” would not result in systems that cou

.00 (c) 1999 IEEE 6

cea thoat,.)s

ndion

laa

an

inmt f

m

ch

t

usoa

s toedf

hethhese

otn

r-cl

n o

elpe

ce

fheo be ofe oflti-dod to

thisedodeated

toomupsentral

,to

of

taski

gnce.

andce-o-

nd

ally

ofort

al

up


be used for a wide range of task types and circumstanand desirable outcomes. Part of the research agendGSS in the future must be to tackle head-on some of glaring weaknesses. How can the poor results fsubjective satisfaction be improved, for instance? Whmanipulations in how the systems are used (toolstraining, anonymity, process structures, facilitation, etccan substantially improve the ratio of positive proceseffects to negative process effects?

The implications of the modal finding of “nosignificant difference” are somewhat different for decisioroom GSS as compared to distributed CMC basesystems. It makes no rational sense to build a decisroom and bring people there to use it if the hardware asoftware will not produce results that are significantdifferent than would be obtained without using such system. However, distributed meetings using CMC mbe chosen by groups even if there is often “no significdifference” in process and outcomes compared totraveling to a FtF meeting, because of the savings travel time, convenience, and related costs. To soextent, one can turn our results around and say thadistributed groups using CMC based decision supporsystems, results are likely to be at least as good or bettethan the results they would obtain by expending the tiand money to travel to a meeting. This argues for furthedevelopment of distributed asynchronous systems, whifit in well with the work styles of already “wired”professionals, who use email and the World Wide Webaccomplish many of their non-meeting work tasks.4.1 What Needs to be Studied?

Interacting social groups cannot function if they foconly on “goal” or "task" oriented activities; they must alsdevote time and interactions to the social-emotioninteractions which build cohesion and trust and thuprovide the group with the resources that enable itsucceed. As Bales put it [1, p. 8], “The idealizinteraction process would then be described as one oalternating emphasis on the two types of problems. Wattention is given to the task, strains are created in social and emotional relations of the members of tgroup, and attention then turns to the solution of theproblems. So long as the group devotes its activity simplto social-emotional activity, however, the task is ngetting done, and attention would be expected to turagain to the task area." Group Support Systems are builtexplicitly to provide task support. Text based computemediated communication is not as "rich" as face-to-faor multi-media interactions for social-emotionacommunication (e.g., positive, such as joking andshowing satisfaction or rewarding others with attentioand approval; or negative, such as showing tension

0-7695-0001-3/99 $10

sfore

nd

t

eo

e

o

l

ne

e

hostility). Thus, it may be that overall we have "nodifference" as the average group result in comparing face-to-face and GSS text based media because the tools hin the task oriented interactions, but groups havdifficulty in CMC with the relative lack of socialpresence, particularly if they have little prior experienwith the system and no history as a group [20].

Many recent studies are examining the effects odifferent GSS structures and tools, which is beyond tscope of this paper. But, clearly, those issues need taddressed. Of particular interest is the open questionwhether new GSS systems integrating the ease of usthe "point and click" web based interfaces and mumedia features will improve the functionality anusability of GSS systems, and thus increase the likelihoof positive outcomes in comparison to unsupported faceface groups. Based upon the results reported in paper, it would make sense to investigate combinmodes of communication and decision making. Fexample, on a more complex task that requires both igeneration and decision making, groups might be starin a decision room using face to face interaction to "getknow" one another. They might then use a decision roGSS to generate ideas. Then, using CMC, the grocould gather more information in order to assess differoptions and reach a solution to the problem. Seveexperiments like this are in progress (e.g., OckerFjermestad, Hiltz, and Turoff, [29]. The results do tend support a combined mode approach.AcknowledgmentsPartial funding was provided NSF-IRI-9015236 and theNew Jersey Commission on Science and Technology.References[1] Bales, R.F., Interaction Process Analysis, Chicago: UniversityChicago Press, 1950,[2] Bui, T., and Sivasankaran, T.R. Relation between gdss use and groupcomplexity: an experimental study. Proceedings of the Twenty-Third HawaiInternational Conference on System Sciences, 3(1990), 69-78[3] Bui, T.; Sivasankaran, T.R.; Fijol, Y.; and Woodbury, M.A. Identifyinorganizational opportunities for gdss use: some experimental evideTransactions of the Seventh International Conference on Decision SupporSystems, (June ,1987), 68-75[4] Chidambaram, L., and Jones, B. Impact on communication medium computer support on group perceptions and performance: a comparison of faface and dispersed meetings. MIS Quarterly 17,4 (1993), 465-491[5] Connolly, T.; Jessup, L.; and Valacich, J.S Effects of anonymity aevaluative tone on idea generation in computer-mediated groups. ManagemenScience 36, 6 (1990), 689-703[6] Daily, B.; Whatley, A.; Ash, S.R.; and Steiner, R.L. The effects ofgroup decision support system on culturally diverse and culturahomogeneous group decision making. Information & Management, 30,(1996), 281-289.[7] Dennis, A.R.; Haley, B.J.; and Vandenberg, R.J. A meta-analysiseffectiveness, efficiency, and participant satisfaction in group suppsystems research. Proceedings of the Seventeenth InternationConference on Information Systems, 1996, 278-289.[8] DeSanctis, G. and Gallupe, R.B. A foundation for the study of grodecision support systems. Management Science, 33, 5 (1987), 589-609.

.00 (c) 1999 IEEE 7

s in

m

ce

Usen

ms

siooloon

po

o

98)

te

s

te

em

ppo

eesarc

: ESa

f gsups

y o

tes a

ted

er-l

o

s,

anen

ivity

tory

ofpment

emsl

oupk, J..

of

ms

e

r

sk

nce

il's

oup

tual

sical

uiryerbal

up

asision

and

y in


[9] DeSanctis, G. and Poole, S. Understanding the differencecollaborative systems use through appropriation analysis. Proceedings ofthe Twenty-Fourth Annual Hawaii International Conference on SysteSciences, 3, (1991), 750-757.[10] DeSanctis, G. and Poole, S. Capturing the complexity in advantechnology use: adaptive structuration theory. Organizational Science, 5, 2(1994), 121-147.[11] Easton, G.K.; George, J.F.; Nunamaker, J.F; and Pendergast, M.O. two different electronic meeting system tools for the same task: an experimcomparison. Journal of Management Information Systems 7, 1 (1990), 85-99[12] Fjermestad, J. An integrated framework for group support systeJournal of Organizational Computing and Electronic Commerce, 8,2(1998), 83-107.[13] Fjermestad, J. and Hiltz, S.R. Experimental studies of group decisupport systems: an assessment of variables studied and methodProceedings of the Thirtieth Annual Hawaii International Conference System Sciences, 2, (1997), 45-66.[14] Fjermestad, J., and Hiltz, S.R., “An assessment of group supsystems experimental research: methodology and results. Journal ofManagement Information Systems, in press, (1999).[15] Fjermestad, J. and Hiltz, S.R. An analysis of the effects of modecommunication on group decision making Proceedings of the ThirtiethAnnual Hawaii International Conference on System Sciences, 1, (1917-26.[16] Fjermestad, J et. al.. Distributed computer supported cooperative stradecision making using structured conflict and consensus approaches. Proceedingsof the Twenty-Eight Hawaii International Conference on System Science3,(1995), 222- 231[17] Fjermestad, J. and Seah, T.S. “Using an executive information systo analyze group support systems: a preliminary implementation,”Proceedings of the Third Americas Conference on Information Syst,(1997), 563-565.[18] Gallupe, R.B.; DeSanctis, G.; and Dickson, G.W. Computer-based sufor group problem-finding: an experimental investigation. MIS Quarterly (June,1988), 277-296[19] Griffth, T.L., and Northcraft, G.B. Distinguish between the forest and trmedia, features, and methodology in electronic communication reseOrganizational Science, 5, 2 (1994), 272-285[20] Hollingshead, A.B. and McGrath, J.E. Computer-assisted groupscritical review of the empirical research. In Guzzo, R.A. and Salas,(eds.), Team Effectiveness and Decision Making in Organizations, Francisco: Jossey-Bass, 1995..[21] Huang, W.; Raman, K.S.; and Wei, K.K. A process study of effects oand task type on informational and normative influence in small groProceedings of the Fourteenth International Conference on InformationSystems, (1993), 91-101[22] Jessup, L.M.; Connolly, T.; and Galegher, J. The effects of anonymitgdss group process with an idea-generating task. MIS Quarterly (September,1990), 313-321[23] Jessup, L.M.; Egbert, J.L; and Connolly, T. Understanding compusupported group work: the effects of interaction frequency on group procesoutcome. Journal of Research on Computing in Education, 28, 2, (1996), 190208[24] Jessup, L.M., and Tansik, D.A. Decision making in an automaenvironment: the effects of anonymity and proximity with a group decisionsupport system. Decision Sciences22, (1991), 266-279[25] McLeod, P.L., A Comprehensive model of anonymity in computsupported group decision-making,” Proceedings of the Eighteenth AnnuaInternational Conference on Information Systems, (1997), 223-234.[26] McLeod, P.L., and Elston, S.E. Persuasive linguistic devices in anonymvs identified computer supported groups: an exploratory study. Proceedings othe Twenty-Eight Hawaii International Conference on System Science,(1995), 163-170[27] Mejas, R.; Shephard, M.M.; Vogel, D.R.; and Lazaneo, L. Consensusperceived satisfaction levels: gss and non-gss outcomes within and betweUnited States and Mexico. Journal of Management Information Systems, 13, 3(1997) 137-161

0-7695-0001-3/99 $10

d

ingtal

.

ngy.

rt

f

,

gic

m

s

rt

:h.

a.,n

s.

n

r-nd

us

4

d the

[28] Mejias, R.J.; Vogel, D.R.; and Shepherd, M.M. GSS meeting productand participation equity: a U. S. & Mexico cross-cultural field study. Proceedingsof the Thirtieth Hawaii International Conference on System Sciences, 2, (1997)469-478[29] Ocker, R.; Fjermestad, J.; Hiltz, S.R.; and Johnson, K. An exploracomparison of four modes of communication for determining requirements: resultson creativity, quality and satisfaction. Proceedings of the Thirtieth AnnuaHawaii International Conference on System Sciences 2, (1997), 568-577[30] Ocker, R.; Hiltz, S.R.; Turoff, M.; and Fjermestad, J. The effectsdistributed group support and process structuring on requirements develoteams: results on creativity and quality. Journal of Management InformationSystems, 12, 3, (1996), 127-153[31] Poole, M.S., and DeSanctis, G. Use of group decision support systas an appropriation process. Proceedings of the Twenty-First AnnuaHawaii International Conference on System Sciences, 2, (1989), 149-157.[32] Poole, M.S., and DeSanctis, G. Understanding the use of grdecision support systems: the theory of adaptive structuration. In Fuland Steinfield, C. (eds.), Organizations and Communication TechnologyNewbury Park, CA: Sage, 1990.[33] Sambamurthy, V. and DeSanctis, G. An experimental evaluationgdss effects on group performance during stakeholder analysis. Proceedingsof the Twenty-Third Annual Hawaii International Conference on SysteScience, 3, (1989), 79-88.[34] Sia, C.; Tan, B.C.Y.; and Wei, K.K. Exploring the effects of somdisplay and task factors on gss user groups. Information & Management,30, (1996), 35-41.[35] Simmel, G. Soziologie, Untersuchungen uber die Formen deVergesell Schaftung, Leipzig: Duncker and Humblot, 1892, 1907.[36] Simmel, G., translated and edited by K.H. Wolff. The Sociology ofGeorg Simmel, New York: The Free Press, 1950.[37] Tan, B.C.Y.; Raman, K.S.; and Wei, K.K. An empirical study of tadimension of group support systems. IEEE Transactions on Systems, Man, andCybernetics, 24, 7 (1994), 1054-1060[38] Tan, B.C.Y.; Wei, K.K.; and Watson, R.T. Dampening status influeusing a group support system: an empirical study. Transactions of the ThirteenthInternational Conference on Decision Support Systems (1993), 77-89[39] Tung, L.L., and Heminger, A.R. The effects of dialectical inquiry, devadvocacy, and consensus inquiry methods in a gss environment. Information Management, 25, (1993), 33-41[40] Valacich, J.S.; Dennis, A.R.; and Nunamaker, J.F. Anonymity and grsize effects in an electronic meeting systems environment. Small Group Research23, 1 (1992), 49-73[41] Valacich, J.S.; Dennis, A.R.; and Nunamaker, J.F. A concepframework of anonymity in group support systems. Group Decision Negotiation, 1, 3 (1992), 219-241.[42] Valacich, J.S.; George, J.F.; Nunamaker, J.F.; and Vogel, D.R. Phyproximity effects on computer-mediated group idea generation. Small GroupResearch, 25, 1 (1994), 83-104[43] Valacich, J.S., and Schwenk, C. Devil's advocacy and dialectical inqeffects on group decision making using computer-mediated versus vcommunication. Organizational Behavior and Human Decision Processes 632 (1995), 158-173[44] Watson, R.T.; Ho, T.H.; and Raman, K.S. A fourth dimension of grosupport systems. Communications of the ACM, 37, 10 (1994), 44-55[45] Wheeler, B.C., and Valacich, J.S. Facilitation, gss, and trainingsources of process restrictiveness and guidance for structured decmaking an empirical assessment. Information Systems Research, 7, 4(1996), 429-450.[46] Wilson, J., and Jessup, L.M. A field experiment on gss anonymity group member status. Proceedings of the Twenty-Eighth Hawaii InternationaConference on System Sciences, 4, (1995), 212-221[47] Wood, J.G., and Nosek, J.T. Discrimination of structure and technologa group support system: the role of process complexity. Proceedings of theInternational Conference on Information Systems 3, (1994), 187-199[48] Zigurs, I. and Kozar, K.A., “An Exploratory Study of Roles inComputer-Supported Groups,” MIS Quarterly, 18, 3, 1994, 277-297.

.00 (c) 1999 IEEE 8


Tab

le 1

FtF

vs.

GS

S A

sses

smen

t Res

ults

: Cou

nts

for

All

Exp

erim

ents

on

Inde

pend

ent V

aria

bles

(Un

it of

Me

asu

rem

en

t is

the

Hyp

oth

esi

s)In

depe

nden

tT

ota

l Of

No

Pos

itive

Neg

ativ

eN

oO

the

rP

erce

nt

Ra

tiV

ari

abl

eC

oun

tS

ign

ifica

nt

Effe

ctE

ffect

Ma

in E

ffect

Effe

cts

&P

ositi

veP

ositi

ve/

Ca

tego

ries

Effe

ctG

SS

> F

tFF

tF >

GS

SM

easu

res

Inte

ract

ion

Effe

cts

Neg

ativ

eE

ffect

sE

ffect

sT

echn

olo

g

Tas

k Su

pp

ort

4118

23..

..

Pro

cess

Str

uct

ure

380

177

135

1217

36.

72.

6

Des

ig44

1925

....

Com

mu

nica

tion

531

198

139

139

2134

29.2

1.0

Tec

hno

logy

To

tal

996

412

152

144

3325

521

.51.

1G

roup

223

972

431

891.

80.

5T

ask

214

561

1186

601.

50.

1C

onte

xt13

862

35

3533

4.3

0.6

Met

hod

111

91

100.

01

Tot

als

1582

628

158

164

194

438

16.6

1.0

0-7695-0001-3/99 $10.00 (c) 1999 IEEE 9


Tab

le 2

FtF

vs.

GS

S A

sses

smen

t Res

ults

: Cou

nts

for

All

Exp

erim

ents

on

Dep

ende

nt V

aria

bles

(Un

it of

Me

asu

rem

en

t is

the

Hyp

oth

esi

s)D

epen

dent

Tot

alN

oP

ositi

veN

egat

ive

No

Oth

erP

erce

ntR

atio

Var

iabl

eC

ount

Sig

nific

ant

Effe

ctE

ffect

Mai

n E

ffect

Effe

cts

&P

ositi

veP

ositi

ve/

Cat

egor

yE

ffect

GS

S >

FtF

FtF

> G

SS

Mea

sure

sIn

tera

ctio

nE

ffect

sN

egat

ive

Effe

cts

Effe

cts

Ada

ptat

ion

Fac

tors

Str

uctu

ratio

n46

176

19

1325

.06.

0P

roce

ss V

aria

bles

146

6118

1718

3218

.81.

1P

roce

ss Is

sues

4416

48

97

14.3

0.5

Pro

cess

Ga

ins

142

6214

1412

4015

.61.

0P

roce

ss L

osse

s71

269

109

1720

.00.

9R

ole

Out

com

es22

52

44

718

.21.

0

Sub

Tot

al

471

187

5354

6111

618

.01.

0

Out

com

e F

acto

rsE

ffici

enc

9722

1026

1821

17.2

0.4

Effe

ctiv

enes

s61

723

073

4475

195

21.0

1.7

Sat

isfa

ctio

n28

013

318

2933

6710

.00.

6C

onse

nsu

s67

303

63

257.

70.

6U

sabi

lit42

251

34

93.

40.

3

Sub

Tot

al

1103

440

105

108

133

317

16.1

1.0

Inte

rven

ing

Fac

tors

Sum

ma

ry V

ari

able

s8

12

5..

..

Tot

al

1582

628

158

164

194

438

16.6

1.0

0-7695-0001-3/99 $10.00 (c) 1999 IEEE 10

An assessment of group support systems research: results

Documents

Transcript of An assessment of group support systems research: results