Idea-Generation in Naturally Occurring Contexts: Complex Appropriation of a Simple Group Procedure

560 HUMAN COMMUNICATION RESEARCH / October 2003

Idea-Generation in Naturally OccurringContextsComplex Appropriation of a Simple Group Procedure

MICHELE H. JACKSONUniversity of Colorado at BoulderMARSHALL SCOTT POOLETexas A&M University

This study examined naturally occurring idea-generation in organizational groups completingan extended problem-solving task. Meetings held by 11 groups engaged in a qualityimprovement process in a governmental agency were analyzed to identify 37 idea-generationepisodes. All groups had available a group decision support system (GDSS), although someopted not to use it. Across all groups, idea productivity was low though fairly efficient, butGDSS-supported idea-generation produced significantly fewer ideas. In general, idea-generationas a process appears to be more complex than has been thought; observed deviations from idealizedidea-generation norms were not necessarily dysfunctional. The findings encourage reassessmentof the assumptions underlying existing normative models of idea-generation.

The process of drawing together ideas from individual group mem-bers into one list serves important functions across many groupactivities, including problem solving, decision making, negotia-

tion, planning, and innovation. In most contemporary group research,idea-generation is regarded as a distinctive creative task that should beguided by procedural norms and undertaken during a bounded periodof interaction (e.g., Ellis & Fisher, 1994; Jarboe, 1996; McGrath, 1984;Pinsonneault, Barki, Gallupe, & Hoppen, 1999; Poole & Baldwin, 1996).

The procedural norms for idea-generation aim to maximize the numberand quality of ideas and to create an open and participative climate in

Michele H. Jackson (Ph.D., 1994, University of Minnesota) is an assistant professor in theDepartment of Communication at the University of Colorado at Boulder. Marshall ScottPoole (Ph.D., 1980, University of Wisconsin-Madison) is a professor in the Department ofSpeech Communication, Texas A&M University. Preparation of this article was supportedby National Science Foundation Grant SES-8715565 and by a grant from the University ofColorado. Any conclusions drawn here are solely those of the authors and do not reflect theopinions of the grantors. A previous version was presented at the 2001 conference of theNational Communication Association. We thank Sian Owen-Cruise for assistance in cod-ing, Gerardine DeSanctis for discussions during the planning of this study, and April Trees,Amy Grim, and Nancy Mann for their comments and suggestions. Correspondence con-cerning this article should be addressed to Michele H. Jackson, Department of Communica-tion, UCB 270, University of Colorado, Boulder, CO 80309–0270; email:[email protected].

Human Communication Research, Vol. 29, No. 4, October 2003 560–591© 2003 International Communication Association

Jackson, Poole / IDEA-GENERATION IN NATURALLY OCCURRING CONTEXTS 561

the group. These norms are embodied in many of the techniques devel-oped by researchers and practitioners to structure idea-generation ingroups (see Jarboe, 1996), including brainstorming (Osborne, 1957), theNominal Group Technique (NGT; Delbecq, Van de Ven, & Gustafson, 1975),brainwriting (Geschka, Schaude, & Schlicksupp, 1975), excursion(Gryskiewicz, 1987), and others (e.g., Nutt, 1984). Each of these proce-dures calls for group-based idea-generation to be conducted within a fa-cilitated or otherwise controlled situation, in which one or more centralpersons guide and structure the group process. Researchers have alsoexplored the role of technology in fulfilling these functions (see the sum-mary in Pinsonneault et al., 1999).

In practice, groups generate ideas in a range of situations, from thoseformally planned to those arising informally during the course of com-pleting some larger task. Time pressure, conflicts, hidden agendas, mis-takes, and misunderstandings of procedures may lead a group to departfrom ideal models of idea-generation. However, how serious these de-partures and their implications are remain largely unexplored. This studyseeks to examine how idea-generation procedures that groups are taughtare applied “in the wild.”1

The theory of structuration offers a useful perspective for understand-ing how and why changes in procedures come about when they are putinto practice (Browning & Bayer, 1998; DeSanctis & Poole, 1994; Jackson,Poole, & Kuhn, 2002; Poole, Seibold, & McPhee, 1985; see Poole, Seibold,& McPhee, 1996, for a review). From the standpoint of structuration theory,procedures are structures that the group appropriates to create a “struc-ture-in-action.” This localized version of the structure may differ—in smallor major ways—from the “institutionalized,” generally shared procedurethat is codified in texts and taught in classes and training seminars. Forexample, the group may have difficulties in reconciling the structure ofidea-generation procedures with other structures that are shaping its in-teraction (e.g., the larger task to which the idea-generation episode con-tributes, or other procedural norms that have evolved in the group). Asecond reason groups may vary is that idea-generation may serve func-tions other than that suggested by its stated goal. If members employbrainstorming as a ritual to unify the group and show that it is followingproper procedure, then they may expect to stick to procedures religiously.On the other hand, if one member uses brainstorming to manipulate thegroup into accepting a preferred alternative, the procedures will likely bealtered so as to slant the playing ground to facilitate that member’s con-trol of the group. Finally, the context may cause groups to deviate fromideal procedures. If the group has little time, works in a noisy room, or isunder pressure from its manager, members may be so distracted that theydo not attend to the norms.


This study represents an attempt to go beyond the obvious conclusionthat procedures differ in theory and in practice to determine the degree towhich norms change and how groups produce and reproduce the nor-mative structure. To this end, we conducted a microlevel analysis of in-teractions during a set of idea-generation episodes occurring in 11 qual-ity improvement groups in a major government organization. The set ofepisodes includes examples of formal idea-generation convened and ledby a facilitator, as well as informal, unplanned idea-generation arisingduring the course of group discussion. The episodes varied in length andin method used for recording ideas. Some groups used a flipchart, otherspaper, and still others a computerized group decision support system (GDSS).

We coded these episodes and compared the actual interaction patternsto those that would be expected from an ideal-type idea-generation pro-cedure or structure. The results provided useful descriptive informationconcerning how idea-generation occurs in ongoing groups. The first sec-tion of this report identifies four fundamental principles in the standardconception of idea-generation found in the literature. We then state re-search questions, lay out the methodology of the study, present results,then discuss implications.

IDEA-GENERATION AS A GROUP ACTIVITY

The core elements of most techniques for idea-generation in groupsgenerally follow Osborne’s (1957) classic concept of brainstorming andits four procedural guidelines: (a) no criticisms or judicial judgment ofideas, (b) free-wheeling, wild ideas are welcomed, (c) quantity is wanted,and (d) combination and improvement of ideas are sought (pp. 300–301).The many techniques developed since 1957 vary in their specific guide-lines for carrying out idea-generation. However, at least four principlesremain roughly constant across the techniques.

First, the goal of idea-generation is to generate as many ideas as possible. Thebest way to promote creativity and quality is to produce many ideas. Forexample, the brainstorming technique assumes that the first ideas thatcome to an individual’s mind are typically commonplace and ordinary;therefore, one of the best ways to get ideas to bubble to the surface is tospin out as many as possible and hope the good ones work their way out.Gordon’s design synthesis approach built upon this principle, using tech-niques to reduce psychological resistance to idea generation and to tire groupmembers to the point of what he saw as real creativity (Arnold, 1962).

Second, during a session, members should concentrate single-mindedly onidea-generation. Based on formal techniques imposed on group process,idea-generation occurs in episodes that are distinctive, recognizable bothto observers and to the group members themselves. Within each episode,


group interaction should support productive ideation. Comments suchas tangents and criticisms of ideas move the group off task and should bediscouraged or prevented. This may be accomplished via facilitation, asin brainstorming or design synthesis, by removing or severely limitingverbal interactions, as through NGT or brainwriting, or by strictly regi-menting the sequence of discussion, as in the 3M techniques team col-laboration and stop-and-go brainstorming (Mason, 1962).

Third, and perhaps most important, productive ideation depends on howwell the group follows the specific rules of the particular idea-generation tech-nique being used. The principle is inherent in the very idea of techniques(Nutt, 1984). Techniques structure interaction so that it proceeds accord-ing to a set of rules or steps. Indeed, a number of idea generation tech-niques including NGT, brainwriting (Method 635), and brainwriting poolclaim to improve the original brainstorming technique because the newrules are easier for the group to follow (Delbecq et al., 1975; Geschka,1993). For many techniques, rules attempt to maximize individual contri-butions by finessing the paradoxical influence of other group memberswho may have simultaneous positive and negative effects (McGrath, 1984;Steiner, 1972).2

Fourth, idea-generation should be marked by a commitment to a posi-tive and cooperative group climate. Such a spirit can encourage and rewarddivergence and creativity from all members by enabling them all to par-ticipate and validating their contributions (Jablin, 1981; Osborne, 1957;Sunwolf & Seibold, 1999). Cultivating this spirit brings other benefits forthe group, including increased cohesiveness and member commitment.To achieve this spirit, idea-generation procedures attempt to defuse fearof social embarrassment, conformity pressures, status systems, and thelike. At the same time, they attempt to maximize social support and rein-forcement for contributing, cross-stimulation, and so forth (McGrath, 1984).

RESEARCH QUESTIONS AND HYPOTHESES

In an effort to examine these four principles, we proposed the follow-ing research questions and hypotheses. First, from a purely descriptivepoint of view, we can calculate the productivity of the episodes in terms ofnumber of ideas produced (output) and ideas produced per minute of thesession and per member. Hence, we posed the following research question:

RQ1: How productive are natural idea-generation episodes?

Comparing these values to similar values from laboratory studies cangive us some idea of how much teams in this study emphasized production


of ideas. Considering additional factors investigated in other researchquestions, such as the prevalence of tangents in the episodes, may shedsome additional light on the degree to which productivity was empha-sized.

A related question concerns whether productivity varies when addi-tional procedural structure is provided by a computerized group deci-sion support system. Results of previous research are mixed. Several stud-ies indicate that groups using computerized idea-generation tools greatlyoutperformed those using manual media (Daily, Whatley, Ash, & Steiner,1996; Gallupe, Cooper, Grise, & Bastianutti, 1994; Valacich, Paranka,George, & Nunamaker, 1993), others find that groups using these toolsgenerated fewer ideas (Pinsonneault, Barki, Gallupe, & Hoppen ,1999),and still others find no difference either in generated idea quantity(Hollingshead, McGrath, & O’Connor, 1993; Straus & McGrath, 1994), orquality (George et al., 1990). This suggests that the effects of computer-ization on idea-generation may be complex. Given that the spirit of theGDSS used in this study was to facilitate the ability of groups to followthe four principles discussed previously in order to improve productiv-ity, we hypothesized that groups using the system would be more pro-ductive than groups using other recording media. Specifically:

H1: GDSS-supported idea-generation will be more productive than manuallysupported idea-generation.

The remaining research questions focus on how the group structuresits idea-generation activity with reference to the other three principles.Recall that groups may not faithfully follow these principles owing tomisunderstandings, difficulty in reconciling the procedure with their long-term projects or tasks, incompatibility with other norms, or motivationsother than idea-generation. To investigate whether and how idea-genera-tion procedures were structured in group interaction, we will considerthe second, third, and fourth principles in some depth, and then return tothe first principle in the discussion.

From above, the second general principle of idea-generation is thatgroups should focus single-mindedly on ideation. Therefore, a group fo-cused on ideation should exhibit few tangents. The ratio of time spent intangents to time spent in ideation offers a gauge of single-mindedness.The nature of the tangents may also shed light on the relative emphasisthe group places on ideation: Some tangents may be more closely relatedto the group’s work than others. Another indication of single-mindednessis the way in which groups respond to tangents as well as criticisms ofideas. Responses that discourage such acts, or that bring the group effi-ciently back to ideation, provide evidence of single-mindedness.


Single-mindedness is also related to patterns of idea-generation.Mindful of their performance, group members should conscientiouslyavoid stopping midtask to attend to other matters. This picture leads tothe expectation that in more productive groups, ideation will be displayedconsistently throughout the episode and the idea-to-tangent ratio will behigh. Further, members will not elaborate ideas until after all ideas havebeen stated, because lengthy discussions or clarification would impedethe free flow of group thought and discourage building on ideas.

These considerations suggested four specific subquestions necessaryto answer the general question:

RQ2: How single-minded are groups about idea-generation?RQ2a: How much do tangents occur during idea-generation episodes?RQ2b: What is the nature of tangents that do occur during idea-generation

episodes?RQ2c: How do groups handle tangents during idea-generation episodes?RQ2d: What are the patterns of idea expression during idea-generation episodes?

If the group is following the third principle, that productive ideationdepends on how well the group follows the specific rules of the particu-lar idea-generation technique being used, we should find indications thatmembers have adopted some structured approach to idea-generation, ei-ther by mutual agreement or because it has been imposed upon them. Ifthey are aware of idea-generation rules and normative models (in thiscase, the rules of Osborne’s, 1957, model), their actual interactions shouldbe patterned accordingly. An indirect measure of the degree to whichgroups follow norms is the degree to which the proportions of time spentin the various activities are consistent with what would be expected basedon normative models of idea-generation.

Facilitators or leaders (appointed or emergent) are another source ofstructure, insofar as they direct the group and call on norms in the courseof managing the group process. A potentially useful consideration, then,is the source and nature of explicit directions. Is direction used to bring agroup back on task or to “prime the pump” and encourage the group topress for more ideas? Do group members respond positively or nega-tively to direction? More specifically:

RQ3: How much do groups use and enact normative structures during naturalidea-generation?

RQ3a: Do patterns of idea-generation activity indicate that groups are follow-ing the norms of brainstorming?

RQ3b: Does the group actively enforce norms of brainstorming?


The fourth principle is that idea-generation should exhibit a spirit ofcooperation and equality that precludes competition or domination ofthe process by a few members. This spirit is perhaps more important thanadhering to particular structures or rules. Nevertheless, there are rulesthat reflect this spirit: All members are encouraged and expected to par-ticipate, equality of participation is desired, and each idea is consideredof equal importance.

One indicator of whether groups act in a way consistent with this spiritis members’ relative level of participation in idea-generation. Members’level of participation in ideation can be measured and compared to theirlevel of participation in other acts during the idea-generation episode,such as tangents or criticisms. One can also measure how the group re-sponds to criticism, which may cause members to withdraw and stifletheir creativity and openness. Hence, we posed the following researchquestion and related subquestions:

RQ4: How much do groups engaged in natural idea-generation enact a spiritof cooperation and equality and avoid competition and dominance?

RQ4a: Do the group members participate equally in ideation?RQ4b: Do the groups sanction those who criticize ideas?

RESEARCH METHODS AND PROCEDURES

All groups met in the same room, which was equipped with a semi-circle of desks, networked computers, flip charts, and other materials tofacilitate their work. Most teams met about twice a month; their meetingswere videotaped. Between 6 and 25 meetings per team were recorded.The groups could select the specific procedures they used. As a result, themanner in which the idea-generation episodes occurred varied; someteams used the traditional flipchart to record ideas, some had a memberrecord ideas on a piece of paper, some used a computer-supported idea-generation tool, and a few did not record ideas at all. The four recordingmethods presented different types of structures that had to be integratedwith the rules of brainstorming. We identified a sample of 37 idea-gen-eration episodes that occurred during the meetings of the 11 teams thatparticipated in the study. This sample helped us to determine whetherrecording method correlated with differences either in the appropriationof brainstorming procedures or in outputs. The sample also included morethan one idea-generation episode from several of the groups, enabling usto compare idea-generation practices both across and within teams.

Videotapes enabled us to use systematic interaction coding proceduresto explore interaction patterns in some detail. We also developed tran-scripts of the sessions, which allowed us to supplement our analysis ofthe coded data with qualitative observations.


Teams and Research Setting

The 11 teams in the study were part of a joint management–union qual-ity improvement program (JQIP) in a large regional office of a U.S. gov-ernment agency. Members received time off from their regular positionsto participate in teams created to identify a problem and then design andimplement a solution. The teams ranged from 5 to 10 members, with anormal distribution of number of attendees about the mean of 7 membersper meeting. The host organization was quite bureaucratic, and groupmembers generally did not work together outside their quality improve-ment teams, which were cross-functional, cross-departmental, and cross-hierarchical.

The teams received training in Juran’s Quality Improvement Modeland a variety of techniques that supported the Juran model (Juran & Gryna,1980), including brainstorming, typical problem-solving procedures, com-munication skills for working in teams, and methods for collecting andanalyzing data. The training in idea-generation closely followed Osborne’s(1957) original four rules of brainstorming.

In this program, teams had a relatively high level of autonomy andindependence. JQIP facilitators worked with each team and were oftenpresent at meetings, but their job was to guide group process, not con-tent. Teams were self-managing and were free to establish their own lead-ership structure.

A GDSS, Software Aided Meeting Management (SAMM), was avail-able as part of the meeting facility. The system consisted of terminals (dis-plays and keyboards) set low enough for group members to engage inface-to-face conversation and a projector for displaying information on alarge screen. These were connected to a server running the SAMM soft-ware program and located around a horseshoe-shaped table equippedwith chairs that easily swiveled and moved.

The SAMM software was designed to support a variety of group pro-cedural techniques. A different module within the program representedeach of these techniques. For example, in addition to an idea generationmodule (based intentionally on the four principles for idea generationidentified above), the system offered an agenda-setting module and amodule for ranking and rating ideas (see Dickson, Poole, & DeSanctis,1992, for the overall configuration of SAMM). Each module was designedso that SAMM guided the group through the steps of the technique, andduring these steps each group member would interact with SAMM (e.g.,input text or select options) via the keyboard and terminal.

The idea generation module of SAMM allowed members to enter ideasat their terminals. Users entered ideas one at a time; the system processedeach idea, storing it and sending it to a public projected display with nolabel indicating who generated it. The result was a list of “anonymous”


ideas that grew throughout the session. Typically, the idea generation ses-sion involved a period of parallel entry in which all members keyed inideas in silence and then a period of discussion during which the groupwent through the ideas sequentially, editing and adding to the list. How-ever, there were also cases in which one member served as recorder andtyped the ideas as others spoke them.

The functionality described here as basic to the idea generation mod-ule also was provided within other SAMM modules (e.g., agenda set-ting). Some groups would on occasion use one of these other modules foridea generation. Although an unexpected appropriation of SAMM, theprocess was idea generation.

SAMM was the only software to which teams had access during theirmeetings. The software was self-contained, running in a non-windowedUNIX platform; it was not integrated with other types of business soft-ware, such as word processors or spreadsheets. The JQIP facilitators wereavailable to help the teams with SAMM. However, members were en-couraged to take control of SAMM themselves, to incorporate it into theirnormal work routines until it became “second nature” to them, much likeusing a flipchart or pen and paper. SAMM was not envisioned to replaceexisting modes of group communication, but rather to support and en-courage verbal and nonverbal interaction. In this spirit, groups often usedSAMM for short to moderate periods of time and only at certain pointsduring their operations.

Research Procedures

A team of researchers viewed the videotape of each meeting and cata-logued group activities. The members then reviewed the notes to isolateidea-generation episodes within particular meetings. The videotapes weretime-stamped, so the duration of the episodes and their components couldbe coded to the second. Once a meeting segment was identified as a pos-sible idea-generation episode, all subsequent procedures entailed work-ing directly with videotaped interaction rather than with a meeting tran-script.

We defined an idea-generation episode as a period during which thegroup engaged as a collective in the process of producing a list of itemsrelated to the same topic that subsequently would be reduced through aprocess of evaluation. We operationalized a list as at least three items be-cause this is a minimum number for which the idea of a list has face va-lidity. In many cases, the beginning of the episode was signaled by an ex-plicit suggestion that the group should engage in a period of idea-generationor brainstorming. In cases in which there was no explicit suggestion, thebeginning of the episode was marked as the time when the first item of aset of three or more ideas was generated. Episodes ended at the point


when the floor was “closed” to additional ideas; in other words, when nomore ideas were being generated and the focus of discussion shifted to adifferent topic or activity. Two or more researchers (including at least oneof the authors) identified and confirmed episode boundaries before pass-ing the episodes to coders. Because interaction outside team meetingswas not recorded, episodes that extended across two or more meetingswere excluded. Each meeting segment passed to the two researchers cod-ing the data exceeded by 5–10 minutes at each end the correspondingepisode identified through the above process; confirming exact bound-aries of the idea-generation episode was one step of the coding process,explained below. We identified a total of 37 episodes distributed across 11teams.

Two coders (one of whom was one of the authors) then analyzed theepisodes following a multi-step procedure:

1. The coders viewed the entire meeting from which the episode wasexcerpted. The aim was to reach a common understanding of the groupso that they felt they possessed the background needed to code accuratelywhat occurred during the episode. During this viewing, the coders talkedwith one another about general issues connected to (a) the team’s history;(b) the project the team had elected to work on; (c) the agenda and activi-ties engaged in during this meeting, including any problems or conflictsthe team experienced; (d) jargon and other specialized terms related tothe project or the organization; and (e) features of SAMM used in themeeting, if any. The coders did not discuss specific content of the meetingassociated with the idea-generation activity categories.

2. The coders then specifically previewed the segment passed to themfrom the previous stages (the identified episode plus 5–10 minutes oneither side) to achieve consensus on the following questions: (a) whattype of ideas were generated during the episode (e.g., problems, solu-tions); (b) what was the intended purpose of the resulting list of ideas(e.g., to clarify underlying causes of a problem); (c) what was the group’snext step (if one could be ascertained from the viewing); (d) were theretechnical difficulties and, if so, how did the group responded to them; (e)what was the larger group task in which the idea-generation was embed-ded; and (f) what were the exact first and last words of the episode? Thecoders also sketched a diagram indicating the position of each groupmember as appearing on the videotape. If the coders could not come toagreement concerning these questions, or if they agreed that an episodewas unusually complex (e.g., embedded within another idea-generationepisode), they completed the remainder of the coding procedures togetherand attained consensus for all coding decisions. One such complex epi-sode was included in this study.

3. For each episode, the coders coded activities according to a schemedeveloped for this study and described in the next section. The coders


were trained in the scheme using several of the episodes; interrater reli-ability was assessed on a subsample of episodes. Episodes used for train-ing were returned to the data set, and each episode was assigned ran-domly to one of the coders.

Observational System and Other Measures

We developed a coding system through iterative analysis of severalepisodes that involved adding and refining categories and rules to cap-ture data that would enable us to answer the research questions. Once thesystem was complete, all episodes used in the development process wererecoded for the final version.

The unit of analysis for the coding scheme was the action, defined as aperiod of time during which one or more members engage in one of thefollowing five activity categories: ideation, elaboration, criticism, direc-tion, or tangent. Coding was exhaustive: Using one-second intervals, cod-ers analyzed all time in the episode for the occurrence of these five activi-ties (as indicated by either verbal or nonverbal behavior). Actions couldco-occur because different members could engage in different activitiesat the same time (e.g., some members engaging in “side conversations”while another generating an idea). Coders indicated on the timeline, tothe closest second, when each activity began and ended. The coding pro-cedure also recorded which individuals in the group participated in eachaction segment, a brief summary of the action content, and additionalinformation on the characteristics of the action, as indicated in Table 1(see the Appendix for coding form).

Full definitions of the five activity categories are provided in the Ap-pendix. Ideation is the explicit, spoken contribution of an item to be re-corded on the shared list. A special code, generation, was used to identifythose periods during which groups engaged in silent idea-generation, asduring the use of the GDSS. Elaborations clarify or rephrase ideas. Criticismsare negative statements or judicial judgments about proffered ideas. Tan-gents are interactions that are off-topic and break the “singlemindedness” ofthe idea-generation, whether or not they relate to other group work tasks.Finally, directions guide or structure the idea-generation activity.

In addition to coding the activities, we collected data relating to themethod used to record ideas and the number of technical problems, ifany, that occurred during idea-generation.

Because our units of analysis were identified at the same time theywere categorized, and because length of each unit was coded to a stan-dard one-second interval, we combined assessment of unitizing and cat-egorizing reliability. Codes were counted as in agreement if the two codersassigned the same activity to the same period of time, to within 5 sec-onds. Reliability was assessed only for the main activity categories


because these were the only categories that occurred often enough to gen-erate data adequate for the calculation. Cohen’s kappa for interrater reli-ability was .73.

Data Analysis

From the coded data we derived three types of information: (a) time inseconds spent in activities, (b) number of periods (regardless of length) ofeach activity that occurred in an episode, and (c) the timeline with itsordering of actions. The time data enabled us to measure actual activityrates, while the period data indexed how often the group undertook anactivity. The timeline permitted us to explore the sequencing of activitiesthroughout the episodes and derive phases using procedures describedin Poole and Roth (1989).

Time data were corrected for differences in group size by dividing themean scores by the number of members in the group. Period data werederived from the timelines by counting the number of times an activitywas initiated during the episode. If two or more time segments of the sameactivity occurred consecutively, they were counted as a single period.

To measure the degree to which group activities were concentrated ina few members or distributed among all, we calculated an equality index,which measured how far the distribution of participation in an activity

TABLE 1Additional Information Collected for Each Activity

Idea

Elaboration

Criticism

Direction

Tangents

Activity Additional Information

Recorded before or after generation?Recorded by which member?Linked to any other idea?Generated anonymously?Later rejected?

Clarification or rephrasing?

How responded to by rest of group?Effect on the idea?

Purpose of direction?Action taken in response to direction?Solicited?

Type of tangent?How ended?Presence of laughter?Cut off idea?


deviated from absolute equality. Absolute equality was defined as eachmember making an equal number of contributions.3

We used statistical tests where appropriate, but the nature of the sampleplaced some constraints on their applicability. First, owing to the labor-intensive data collection procedures, we had a relatively small sample ofsessions. This limited the power of our statistical tests in some cases.Observed power, reported below, ranged from .17 to .94 for the tests weconducted. While the statistical tests are informative, in view of the factthat this is an exploratory study, we will also interpret mean values. Asecond constraint on the use of statistical analysis was that more thanone episode from each group was included in the sample, which intro-duce a possible group effect. To correct for this, we nested groups withincells in all analyses. We employed a mixed-model ANOVA for the statis-tical tests in this case, because group was a random variable. Third, wewere able to calculate reliability only for the major activity categories. Asa result we performed statistical tests only for these variables. We havereported counts and percentages for finer grained categories such as thesubtypes of tangents, but we did not perform statistical tests for thesedata, which are essentially qualitative. Qualitative data used in this studyalso included documents produced by the groups, interviews with groupmembers and facilitators, and detailed observations of multiple video-taped group meetings.

RESULTS

Descriptive Findings: What Happened in the Episodes?

The average duration of an idea-generation episode in this sample was1308 seconds, about 22 minutes; the shortest episode ran 170 seconds andthe longest 3934 seconds (about 65 minutes). As might be expected, num-ber of ideas correlated significantly with length of idea-generation epi-sode (r = .52, p = .001).

Groups used several different methods to record ideas: the idea-gen-eration module of SAMM in 15 episodes, a flipchart in 11, an ordinarypiece of paper in 8, and no recording method at all in 3. That is, in 11 ofthe episodes (30%), groups did not use a medium that allowed all mem-bers to view the list of ideas easily, which deviated from the rules inwhich they had been trained. Groups also elected to use the GDSS infewer than half the episodes.

Table 2 shows statistics relating to the time devoted to various activi-ties during the idea-generation episodes, corrected for number of mem-bers in the team. More than one member could be involved in any givenactivity and activities could co-occur; as a result, the sum of the times


spent by members in activities could exceed the actual clock time of theepisode, which is indicated in the bottom row of the table. In fact, thesum across the five activities per member was 224 seconds per member,greater than the total episode time per member of 181 seconds. This indi-cates simply that there was a considerable amount of time in which twoor more members shared the floor.

Ideation. The mean number of ideas generated per episode was 15, witha standard deviation of 11.5. Number of ideas ranged from a minimum of3 to a maximum of 53, with a mode of 7 ideas per episode; 25 out of 37episodes generated between 5 and 20 ideas. Group members spent a meanof 71 seconds in actually stating ideas (about 40% of the mean total epi-sode time per member), but this figure varied considerably (SD = 73.38seconds). In some of the episodes, groups that used the GDSS actuallyspent 0 seconds of interaction time vocalizing their ideas; instead, themembers keyed their ideas in and devoted the remainder of their epi-sodes to other activities listed in Table 2.

Number of ideas was not significantly correlated with group size (r =.085, p > .65). In all but the GDSS-supported condition, an idea was talliedif it was vocalized, regardless of whether it was recorded. We perceivedno instances in these conditions in which an idea was recorded but notvocalized. In GDSS-supported episodes, all members keyed in ideas.Because SAMM stored all ideas, the number of ideas produced in theseepisodes was determined from printed SAMM reports. For non-GDSSepisodes an average of 4.5 members contributed to idea-generation,out of an average team size of 7.08. As would be expected, there was astrong relationship between the number of people contributing to idea-generation and the total amount of time spent by the group in idea-generation (r = .59, p < .001), but the relationship was nonlinear. The meanamount of time spent in idea-generation was equal for the 3rd and 4th

TABLE 2Mean Seconds Per Member Spent in Various Idea-Generation Activities

Ideation 71.20 0 347.00

Elaboration 21.89 0 107.33

Criticism 3.00 0 16.40

Direction 16.04 .33 107.30

Tangent 111.98 9.00 426.20

Sum of Activity Means 224.11 — —

Episode Time 180.99 27.00 545.86

Activity Mean Minimum Maximum


quartiles of the number of participants, suggesting that as number ofmembers participating increased, ideas per member decreased. This isprobably due to restrictions on floor time as group size increased.

Elaboration

The mean number of elaboration periods across episodes was 7, with aminimum of 0 and a maximum of 24. Elaboration was common; in onlyfive episodes were there no elaborations. However, not all ideas wereelaborated: In 66% of the episodes only 3 to 6 ideas were elaborated, outof an average 13.08 ideas per episode. The most common type of elabora-tion was clarification (69%), in which members add detail to an idea orrefine its statement. Rephrasing, in which members substantially reworkan idea before adding it to the common list, constituted 19% of the elabo-ration periods. Note, however, that while elaborations are defined hereas occurring immediately after an idea was expressed, additional devel-opment of ideas did occur in one of the tangent types discussed below.

Criticism

Participants spent only a small amount of time criticizing ideas, anaverage of only 3 seconds per member per episode and 20.35 secondstotal per episode. Criticism occurred in only 15 episodes. The mean num-ber of periods of criticism was 1, with a minimum of 0 and a maximum of4. Most of the criticisms occurred relatively late in the sessions, in eitherthe second or third thirds of the episodes. Of 37 total criticisms during the15 episodes, 28 occurred during the last two thirds of the episode.

Tangents

The most time-consuming activities in the episodes were tangents, withan average of almost 2 minutes of tangents per member. The mean num-ber of tangent periods was 17, with a minimum of 3 and a maximum of43. Most episodes had one or more tangents at the beginning, but other-wise there were no regular patterns.

Each tangent action was classified into one of seven categories: (a) an-ecdotes or stories; (b) what Bormann (1985) called “fantasies” (which mightinclude stories about the group, what it would accomplish, and extendedjokes/narratives); (c) discussion of or response to physical distractions;(d) discussions of the group’s process or task in general, includingdiscussion about directions given to the group, and discussion in whichold or new business was raised; (e) revisiting ideas contributed earlier inthe session but unrelated to the idea currently on the floor (discussions ofan idea immediately following its contribution were instead coded aselaboration); (f) discussion of the GDSS in general; and (g) tangents that


occurred during GDSS idea-generation (this pertains only to the com-puter-supported episodes).

Tangents of types a, b, and c represented unproductive distractions.While tangent types d and e distract from the idea on the floor, they wereproductive because they contributed to the group’s progress. Interestingly,these were the two most common tangent types comprising 66% of thetotal tangents. The next most common type was anecdotes (17%), andthen tangents related to the GDSS (13%).

Directions

Across episodes an average of 16 seconds per member were devotedto direction, (equivalent to about 2 minutes of directional activity in aneight-member group), with a minimum of nearly 0 and a maximum of107 seconds. The mean number of periods of direction-giving was 9.2,with a minimum of 1 and a maximum of 23. As might be expected, thegreatest proportion of directions (51%) concerned the idea-generation pro-cess itself. Also prominent were directions about using the GDSS (19% onthe average for the GDSS episodes alone) and requests for directions (13.5%).

ANALYSIS

RQ1: How Productive Are Natural Idea-Generation Episodes?

If one measures productivity as the number of ideas generated, theperformance of groups in these episodes was unremarkable. Groupsproduced as few as 3 ideas in an episode (fewer than one idea per member).The majority of episodes produced 20 or fewer ideas. Table 4 illustratesthe results for ideas generated per member and per minute for thisstudy and selected laboratory studies of idea-generation.4 Groups inthis study generated 3.1 ideas per member in episodes in which

TABLE 3Mean Proportion of Tangent Periods Spent in Various Types of Tangents

Tangent Type

Anecdote .174 .00 .59

Fantasy .022 .00 .20

Other .068 .00 .50

Discussion of General Task .381 .00 .86

Task Related Topics/Revisiting Earlier Ideas .278 .00 1.00

Discussion of GDSS .095 .00 .35

Silent Tangent .034 .00 .33

Mean Proportion Minimum Maximum


flipcharts were used for recording, 1.7 ideas per member in episodesin which ideas were recorded on paper, and 1.8 ideas per member inepisodes in which SAMM was used for recording. All of these numberswere considerably lower than those for laboratory studies: The leastproductive lab groups generated 6.7 ideas per member using a flipchartand 7.2 ideas per member using electronic brainstorming. There weresimilar contrasts for ideas per minute.

Lower productivity did not appear to be due to the fact that the groupsin this study had more members than the laboratory groups. In experi-mental studies, increasing group size tends to depress the number of ideasgenerated; in our study, the smaller groups (5 and 6 members) generated2.3 ideas per member and .75 ideas per minute, close to the means forgroups of all sizes. Compare these to the average of 8 ideas per memberand 3.3 ideas per minute for the 6-person groups in Pinsonneault et al.’s(1999) laboratory study.

H1: GDSS-Supported Idea-Generation Will Be More Productive ThanManually Supported Idea-Generation

Medium does make a difference, at least if we contrast the computer-supported episodes with those using other modes of idea-generation in

This study (5 to 10) 3.1 (manual, flipchart) 1.0 (manual, flipchart)1.7 (manual, paper) .7 (manual, paper)1.8 (computer supported) .6 (computer supported)

Bouchard, 1972 (4) 14.8 (manual) 5.8 (manual)

Jablin, Seibold, 15.1 (manual) 4.0 (manual)& Sorenson, 1977 (4)

Gallupe, Bastianutti, 10.6 (manual) 2.8 (manual)& Cooper, 1991 (4) 12.9 (computer supported) 3.4 (computer supported)

Diehl & Stroebe, 1991 (4) 12.4 (manual, interacting) 3.4 (manual, interacting)17.7 (manual, nominal) 5.0 (manual, nominal)

McKinlay, Procter, 7.4 (manual, interacting) 2.2 (manual, interacting)& Dunnett, 1999 (3) 11.7 (manual, nominal) 3.5 (manual, nominal)

7.2 (computer supported) 2.2 (computer supported)

Pinsonneault, Barki, Gallupe, 6.7 (manual) 2.7 (manual)& Hoppen, 1999 (6) 9.8 (computer supported) 3.9 (computer supported)

TABLE 4Number of Ideas Generated Per Member and Per Minute

for This Study and Six Selected Laboratory Studies of Idea-Generation

Study(Group Size)

Ideas Generated Per Member(Recording Method)

Ideas Generated Per Minute(Recording Method)


this study. We conducted one-way ANOVAs to compare the levels ofidea-generation in episodes in which groups used the GDSS, flipchart,and paper. We did not include in the ANOVAs episodes in which norecording device was used because this condition contained only twogroups. Episodes in which groups used the GDSS generated a mean of 13ideas; those using a flipchart generated a mean of 22 ideas; those usingpaper generated a mean of 11 ideas; and those using no recording devicegenerated a mean of 6 ideas. A one-way ANOVA comparing GDSS,flipchart, and paper groups showed a significant difference, but only atthe .10 level (F3, 16.52 = 2.63, p = .065; eta2 = .302; observed power = .536; theeffect for groups within cells was nonsignificant). A post hoc contrast todetermine whether the flipchart condition generated more ideas than theother two conditions was significant (t = 2.70, df = 26, p = .012 These resultswere counter to those of most laboratory experiments, which show thatcomputer-mediated groups generate significantly more ideas than face-to-face groups.

We conducted a further series of statistical analyses comparing GDSS,flipchart, and paper episodes (the no recording condition had only threeepisodes and so was omitted from these analyses). In terms of total timespent brainstorming, when groups used the GDSS they devoted an aver-age of 1630 seconds to idea-generation; when the flipchart was used groupsdevoted 1372 seconds; and when paper was used groups devoted 911seconds. The ANOVA indicated no significant difference among thesemeans, but there was a significant effect for groups within cells (F15,15 =6.66, p < .001, observed power .99; observed power for nonsignificant ef-fect .23). When groups used no recording device they spent 179 secondson idea-generation. In terms of efficiency, defined as time spent per idea,flipchart episodes were more efficient than GDSS or paper episodes. Theytook 66 seconds per idea, whereas GDSS episodes consumed 149 secondsper idea, and paper episodes 91 seconds. These differences were not sig-nificant, but there was a significant effect for groups within cells (F15,11 =11.22, p < .001, observed power = .99; observed power for nonsignificanteffect = .41).

Table 5 shows the results of ANOVAs for each of the five activities (theobserved power for the tests ranged from .14 to .80). There was a signifi-cant difference in time devoted to directions, and nonsignificant but sug-gestive tendencies for idea-generation, criticisms, and tangents. GDSSepisodes spent more time on idea generation and tangents than did theother two groups. Flipchart episodes showed a tendency to spend moretime on elaboration than did groups using the other modes.

A similar pattern of differences appears in time devoted to the activi-ties per member. Table 6 shows the ANOVAs for each of the five activitiesper member (observed power for these tests ranged from .21 to .49). InGDSS-supported episodes, groups spent more time in directions, but less


in criticisms than the episodes supported by flipchart or paper. Therewere also suggestive tendencies on idea-generation, tangents, andelaborations. Groups in GDSS-supported episodes spent more timeon idea-generation and tangent activities than groups using the othertwo modes. Groups in flipchart-supported episodes spent more timeon elaboration than the other two.

Some differences were observed in number of distinct periods of thefive activities that occurred during idea-generation. The ANOVAs forthe five activities are shown in Table 7 (observed power for these testsranged from .21 to .79).

These tests indicated that the GDSS-supported episodes had more pe-riods of direction than the other episodes. An interesting tendency wasfor flipchart episodes to have more periods of both idea generation andelaboration than the GDSS episodes, suggesting that these groups brokeup idea generation with discussion of ideas.

Ideation 648.7 1212.0 150.1 157.2 127.4 107.8 F2,15.49 = 1.99

Elaboration 202.3 459.5 420.6 390.5 128.0 116.7 F2,15.66 = 1.58

Criticism 10.4 27.5 47.4 91.8 25.4 57.5 F2,18.55 = 0.65

Direction 1125.8 1183.9 108.6 84.3 169.5 134.4 F2,18.96 = 5.63 *

Tangent 737.0 570.9 409.6 210.1 384.5 320.1 F2,16.73 = 2.39

TABLE 5Analysis of Variance for Seconds Spent in Idea-Generation Activities

in the Three Recording Conditions

GDSS Groups Flipchart Groups Paper Groups

Activity Mean SD F TestMean SDMean SD

NOTE: * p = .01.

Ideation 80.3 149.7 23.8 25.9 19.9 14.6 F2,16.76 = 1.12

Elaboration 26.5 64.0 58.1 50.4 19.6 15.9 F2,15.70 = 2.01

Criticism 1.3 3.4 6.4 11.4 4.6 5.7 F2,16.98 = 2.88 *

Direction 135.2 134.0 14.9 10.4 28.2 20.6 F2,17.28 = 2.75 *

Tangent 94.7 66.0 56.6 27.2 65.4 54.4 F2,17.26 = 1.70

TABLE 6Analysis of Variance for Seconds Per Member Spent in Idea-Generation Activities




NOTE: * p < .10.


In sum, groups in GDSS-supported episodes spent more time togenerate fewer ideas than did those using flipcharts or paper; much ofthis time was devoted to managing the GDSS apparatus itself. GDSSsupported episodes also had a tendency to spend more time on tangentsand less on elaboration than did flipchart groups. Overall, GDSS episodestended to have greater standard deviations than episodes for the othertwo modes on all activities except criticism. This seems to account for thefact that there were few significant F-statistics, despite the appearance offairly substantial mean differences between GDSS and either or both ofthe other recording modes. It also suggests that group reaction to GDSSas a mode of recording had greater variability when compared to the othertwo modes.

RQ2: How Single-Minded Are Groups About Idea-Generation?

Generally, ideation counted for only one-fifth of a member’s total ac-tivities, and rarely for more than half. The mean ratio of number of ide-ation activities to total activities was 15% (SD = 14%), with a minimum of0% (this member being a process facilitator who did not contribute ideas)and a maximum of 57%.

One indicator of single-mindedness is a low level of tangents. How-ever, as we noted above, tangents were both frequent and long. Howgroups end tangents gives important information concerning how theyregard them. About 44% of the time, the tangent ended naturally; themember(s) engaged in the tangent brought it to an end, and the groupmoved on. About 23% of the tangents were succeeded by other tangents.In about 30% of the cases, the tangent was interrupted by an effort to getthe group back on track. Approximately 15% of the tangents were inter-rupted by an idea. Directions interrupted 16% of the tangents, but onlyabout 12% of these directions explicitly focused on ending the tangents.The remaining directions were focused on elements of the task, and the

Ideation 4.0 3.4 6.6 1.5 5.4 2.7 F2,15.78 = 1.83

Elaboration 3.1 3.2 5.9 1.8 4.4 2.5 F2,16.17 = 1.59

Criticism .5 1.5 2.0 2.2 2.6 2.1 F2,18.12 = 1.06

Direction 6.5 2.4 5.6 2.3 4.6 1.6 F2,18.44 = 5.56 *

Tangent 6.1 2.2 7.0 1.2 5.4 1.3 F2,16.93 = 1.74

TABLE 7Analysis of Variance for Number of Periods Spent in Idea-Generation Activities




NOTE: * p < .05.


tangent ended as a by-product of the group’s redirection. These resultssuggest that groups did not let tangents divert them from their work inthe majority of instances (77%) and that they often viewed tangents asdistractions.

To characterize patterns of idea-generation in the episodes, we ana-lyzed the timelines using cluster analysis. For the 25 episodes in whichgroups generated more than six ideas, we divided the timelines into fourequal segments and tallied the number of ideas within each segment,which provided a profile of ideas by segments. We then computed dis-tance measures between the profiles of each episode and subjected themto average link cluster analysis. The most plausible solution yielded fourdistinct patterns of idea-generation, with two episodes belonging to nocluster. We verified the cluster solution by using the profile to classifygroups in a classification analysis. The results indicated that 95.7% of theepisodes were classified correctly into the groups assigned by the clusteranalysis. The four sets of episodes exhibited the following patterns:

1. Front-loading. In five episodes, most ideas appeared at the beginningof the session, with the remainder of the episode devoted to directionsand tangents. In two of these, there was also a lesser period of ideation atthe end of the episode.

2. Back-loading. In eight episodes, most ideas appeared at the end of theepisode. Considerable discussion preceded the ideation period.

3. Middle-loading. In six episodes, most ideas emerged in the middle ofthe episode. A substantive amount of discussion appeared both beforeand after most ideation.

4. Even distribution. In four episodes, idea-generation occurred more orless evenly throughout the episode.

In those episodes in which ideas were evenly distributed across theentire episode, we wondered whether ideation had a purposeful quality,or whether the episodes meandered and, thereby, robbed the group ofthe intellectual boost that idea-generation is supposed to supply. We in-vestigated this possibility by exploring ideational density. Ideationally denseepisodes were those in which the number of ideas contributed was equalto or greater than the number of minutes participants spent in activitiesother than idea-generation. By this criterion, all but one of the even-dis-tribution episodes were dense, which suggests that these were not mean-dering.

Overall, the patterns suggested that tangents do not prevent single-mindedness: Single-mindedness does occur within natural groups, butgenerally only within particular segments of the episode.

RQ3: How Much Do Groups Use and Enact Normative Structures Dur-ing Natural Idea-Generation?

Given their training and the fact that brainstorming is a well-known


procedure, group members were expected to have internalized some ofits norms. Also, normative procedures were likely to have particularsalience for these groups, given that they were charged with using aspecific quality improvement process. To what extent, then, did the groupsactually follow these norms?

The groups in this study did seem to follow the norm to refrain fromcriticism and premature judgment of ideas. Criticisms were not com-mon, and occurred in fewer than half the episodes, with no more thanfour periods of criticism within any one episode. Interestingly, the timespent criticizing was positively correlated with the total number of ideas(r = .44, p = .012). This seems to contradict the assumption that criticisminhibits free expression of ideas. However, this result became understand-able when we took into account placement of criticism within the epi-sode. Cases in which criticism occurred during the first third of the epi-sode generally fell at the mean in terms of productivity (mean number ofideas of 14.3, compared to an overall mean of 15.0). In contrast, episodesin which criticism fell in the last two thirds were above the mean in pro-ductivity (mean number of ideas of 20.6).

A second norm the groups were taught was to state ideas quickly andbriefly in order to build a comprehensive list and to delay lengthy expla-nation until after idea-generation was complete. We examined the casesto determine when ideas were stated briefly and succinctly and whenthey were either stated at length (an idea-generation turn of more than 1minute) or elaborated. In only 10 episodes were ideas simply listed withno discussion, and this included the 5 episodes in which all ideas wereentered silently into the GDSS. In 21 episodes, 75% or more of the ideaswere either expressed in statements that consumed more than 1 minuteor accompanied by an elaboration as the next coded action. The prevail-ing practice was to take time to form ideas fully as they were generated,rather than to follow the traditional approach of listing ideas quickly (andperhaps incompletely) and coming back to discuss them later.

A third norm taught was that ideas should be recorded publicly sothat all members could share them and refer to them during discussions.As the results reported above indicate, about 30% of the episodes devi-ated from this norm. All but one of these episodes produced fewer ideasthan the sample mean.

Norms are often enforced by the group’s process facilitator or leader.The number and occurrence of directions in the episodes varied widely,from a single direction to begin idea-generation to extensive directiongiven to GDSS-supported groups concerning how to use the system. Thediscussion of tangents above suggests that, although amount of time spenton directions was strongly correlated with amount of time spent ontangents (r = .538, p = .001), directions were not used explicitly topolice group behavior. Directions that explicitly referred to ending the


tangent actually ended only 12% of tangents. One interpretation is thatgroups with more tangents generally may have needed more guidance tokeep them on track. Directions were devoted primarily to managing theidea-generation process or the computer technology used for idea-gen-eration, not to ending tangents.

RQ4: How Much Do Groups Engaged in Natural Idea-Generation Enacta Spirit of Cooperation and Equality and Avoid Competition and Domi-nance?

The equality index described above ranges from 0 to 1, with a value of0 indicating complete equality of participation in a given activity, and 1indicating concentration of the activity in a single member. Hence, theindex is an inverse indicator of equality. The results for equality did notinclude the five GDSS-supported episodes in which all ideas were en-tered during a period of silent keyboarding by members because theanonymous feature of the GDSS prevented tracing ideas back to the con-tributing member.

As Table 8 shows, tangents, with a mean score of 0.22, were a greatequalizer, but for the other four activities there was a good degree of con-centration. Not unexpectedly, direction-giving evidenced the lowest equal-ity, with a mean score of 0.60. But surprisingly, ideation equality was sec-ond to lowest, with a mean score of 0.51.

Our original question assumed that ideas would be equally distrib-uted across members. As the findings show, they were not. In 5 of the 15GDSS-supported episodes, members contributed ideas solely by typingthem in at their keyboards; in these episodes, all members contributedideas. In nonsupported episodes, we found a much lower distribution ofparticipation, though it is not clear whether members chose not to con-tribute ideas or whether dominant members inhibited them.

Ideation .51 (.046) .09 1.00

Elaboration .37 (.052) .00 1.00

Criticism .46 (.033) .20 1.00

Direction .60 (.037) .27 1.00

Tangent .22 (.023) .08 .89

TABLE 8Equality of Participation in Various Idea-Generation Activities

NOTE: Reference points: 0 = perfect equality; 1.00 = dominance by one member.

ActivityMean Equality Score

(Std. Error) Minimum Maximum


It was also true that some members tended to participate more thanothers across the board. Table 9 shows the correlations among partici-pation rates for the five activities. Nine of 10 correlations were signifi-cant, and several were substantial: Members who contributed a high pro-portion of ideas also tended to engage in higher proportions of criticism,elaboration, and tangents. The one exception was directions; contribut-ing a high proportion of the directions was not related to contributingother activity types, except tangents.

Given that one assumption behind the GDSS was that anonymity fos-ters equal participation, it is of special interest to consider differences forthe various recording methods. Table 10 compares the means for GDSS,flipchart and paper episodes. For both idea-generation and tangents, themean equality in flipchart episodes was higher than that in either GDSSor paper episodes. Post hoc contrast tests indicated that this differencewas significant and that there was no difference between the GDSS andpaper episodes (t = -3.12, df = 26, p = .004 for idea generation; t = -2.31, df= 31, p = .03 for tangents). But interestingly, there was no difference in

Ideation .56 .237 .34 .186 .62 .290 F2,16.77 = 3.34 *

Elaboration .39 .220 .29 .008 .37 .120 F2,14.05 = .937

Criticism .38 .007 .40 .147 .57 .300 F2,5.99 = 1.83

Direction .54 .198 .58 .094 .69 .285 F2,16.04 =.433

Tangent .24 .095 .17 .071 .29 .254 F2,15.31 = 3.72 *

TABLE 10Analysis of Variance for Equality of Participation in Idea-Generation Activities




NOTE: * p < .05.

Ideation .50** .25** .05 .42**

Elaboration .34** .16* .44**

Criticism .19* .30**

Direction .29**

Tangent

TABLE 9Correlations Between Proportionate Engagements of Individual Members

in Various Activities

Ideation Elaboration Criticism Direction Tangent

NOTE: * p = .05. ** p = .01.


participation equality for directions; the means for GDSS and flipchartepisodes were almost equal, and while the mean for paper episodes washigher, the difference was not significant.

Another indicator of whether the groups enacted norms of coopera-tion and participativeness was how they handled criticisms. We classi-fied responses to criticism into several categories that reflected a range ofboth negative and constructive reactions. In 6 cases, criticisms elicitedeither defensiveness or silence (which we can interpret as negative reac-tions). In 10 cases, criticisms led to acceptance and some discussion, whichwe interpret as constructive use of the criticism. In 3 cases criticisms stimu-lated debate, which can also be interpreted as constructive. Hence, in themajority of instances, the groups used criticism constructively. It is alsoimportant to bear in mind that, in general, criticisms were rare. This, too,suggests that participants were following the cooperative norms associ-ated with the ideal model of brainstorming.

DISCUSSION

This study, which presents the first results from a larger project deal-ing with idea-generation in natural groups, makes three contributions tothe research literature. First, its finely detailed descriptions of idea-gen-eration in natural groups suggest that idea-generation is more complex“in the wild” than typically imagined. Second, its structurational perspec-tive encourages us to assess the consequences for the group of deviationfrom the typical procedural norms. Third, its comparison of methods forsupporting idea-generation, with particular focus on group decision sup-port systems, produced findings markedly different from those of labo-ratory studies.

Our most striking descriptive observation is the relatively low pro-ductivity of idea-generation groups in this organization. The differencedoes not seem to stem from training, or recording methods, both of whichwere similar to those used in the lab studies. Two factors may have con-strained productivity. In some groups, quantity of ideas seemed to be lessimportant than other functions that idea-generation served. For example,some groups seemed to use brainstorming ritualistically to mark transi-tions to a new phase of the task. Coming up with a list of ideas seemed tobe merely a way to signal a fresh start; few of the ideas on the list wererevisited in subsequent work. Second, some topics had natural limits onthe number of possible ideas. For example, one group generated a list oforganizational stakeholders that should be included in a survey; stake-holder lists have inherent ceilings.

Idea-generation has traditionally been conceptualized as a fairlystraightforward group process. This study both supports and challenges


idealized descriptions of idea-generation. Viewed through the lens ofstructuration theory, idea-generation is a complex group process in whichmembers adapt normative procedures to the immediate exigencies of thesituation. Group use of procedures is constrained by members’ under-standing of the procedure, the demands of the situation, and interaction.

Overall, group members in this study understood and tried to followbrainstorming norms. They began and ended idea-generation episodesusing explicit statements, which suggests that they recognized idea-gen-eration as a distinct, bounded procedure. They regularly followed direc-tions to guide and structure the process, suggesting that they recognizedthe need to manage it. Within episodes, the group focused and produceda number of ideas in a fairly brief space of time; criticism of ideas wasrare.

On the other hand, about 30% of the time, groups did not record theirideas so that all members could view them. Groups frequently went offon tangents. In over two thirds of the episodes, members devoted sub-stantial time to activities other than idea-generation. On the average, onlyabout 15% of the episode time was actually spent generating ideas. Ideaswere seldom stated briefly and succinctly, but instead were expressed inextended, elaborated statements; in many cases, the group also discussedideas in depth, a clear violation of the stricture against premature discus-sion of ideas. In several groups, members voiced criticisms early in theprocess. These groups expressed fewer total ideas, which indicates thatcriticism may have inhibited members. Participation was not equal; inmost groups, several members did not contribute any ideas, and amongthose who did, often a few contributed most of the ideas and also domi-nated other types of activities.

What accounts for these results? Deviations from standard procedureare not necessarily dysfunctional. The use of brainstorming for topics thathad natural constraints on the number of possible ideas suggests thateven when they know the field of ideas is limited, groups may want toensure that they “cover all the bases.” This may, in turn, explain some ofthe inequalities in participation: Some members may hold back becausethey perceive that all the important ideas are already on the floor. Lengthyinitial statements, immediate elaboration, and “doubling back” in latertangents may mean that group members are not so much creating a list asconstructing shared meaning about each idea and recording it publicly.Tangents that either double back or relate to the larger project suggestthat groups may multi-task and performing steps in parallel, rather thanin strict sequence. Criticisms registered in the last third of an idea-gen-eration episode may not inhibit expression of ideas, but rather help thegroup further work out the significance of their ideas and begin narrow-ing down longer lists of ideas in preparation for future work. Finally, thetime devoted to activities other than idea-generation may indicate that


idea-generation episodes have two different types of boundaries. Anexplicit statement that indicates the floor is open for ideas is a “hard”boundary marking the bid to appropriate the structure of idea-genera-tion. Within the episode, “soft” boundaries set off periods during whichgroups focus on ideation. These types of boundaries in combination con-stitute the idea-generation episode.

Thus, it is possible to conceive of these various behaviors not as viola-tions of procedure, but as appropriations of a social structure (Poole &DeSanctis, 1990). In naturally occurring groups, appropriation patternsought to be influenced by the nature of the group context. There is reasonto explore a model of naturally occurring idea-generation as an “embed-ded group process,” rather than as a single-minded activity. Current mod-els assume that groups “give themselves over” to the process of idea-generation, to the inherent logic of the activity and the assumed value ofproductivity. Yet the variations found among the groups for this studysuggest that groups rarely lose sight of their larger goals; idea-generationprocedures are part of a more encompassing project and so are interwo-ven with nonideational activities that also serve that project. Idea-gen-eration can then be reconceived more richly as a process that may show-case the inventiveness of “ordinary” group interaction.

Further study should therefore explore what groups do with the ideasthey generate. In the sequential normative model, the next step would beevaluation. It would be instructive to compare group behavior duringthese two processes, which groups may collapse in productive ways.

Our groups were drawn from only one organization in the govern-mental sector; future research should examine groups from other con-texts. The problem-solving tasks in this study were complex; simpler pro-cesses might be more conducive to “single-mindedness.” The groups werecross-functional fairly autonomous; though typical within quality-relatedprojects, they are not typical of most ongoing groups in organizations.Finally, it would be important to study contexts in which other technolo-gies are used to support group processes.

In theory, idea-generation is a simple, familiar procedure that lookseasy for groups to appropriate. In actual performance, it is not so simple.Its simplicity as a form then becomes an asset for our larger efforts to sortout the complexity of naturally occurring group interaction.


APPENDIX

Abbreviated Coding Instructions

Task Boundaries

The global boundary of the episode is the period of time in which the group opens thefloor to generating a list of items. The beginning boundary is the point where any memberexplicitly suggests that the group should engage in a period of idea-generation or brain-storming, or, if such a remark is not present, the point when the first idea is generated. Theending boundary is the point where the floor seems officially closed to more items. Up untilthis point, the contribution or elaboration of an idea would still be “in order.”

“First Pass” InstructionsTwo coders complete this pass simultaneously. Coders discuss answers with each other

and reach consensus. If reaching consensus is difficult on questions 2, 3, and 4, the entiretask should be coded by consensus.

1. How are ideas recorded? (If SAMM, what module?) Refer here only to the technologythat is used, not to how it is used. Modules of SAMM: idea-generation, stakeholder analy-sis, agenda, meeting notes. Note if the module cannot be determined.

2. Indicate a category that describes this list of ideas. Possible examples: problems, solu-tions, questions, stakeholders, assumptions.

3. What purpose does this list of ideas serve for the group? How clear is this purpose tothe group (Very, Somewhat, Not at all, Can’t tell)? The purpose of the list is more generalthan a description of the list itself. For example, a group may generate ideas about the causeof a problem, and the purpose might be to clarify or discover underlying issues or factors.Possible purposes: determine next course of action, choose group’s problem, choose solu-tion to follow, clarify underlying issues, develop questionnaire. Be as specific as the evi-dence allows.

4. What is the group’s next step? How clear is this to the group (Very, Somewhat, Not atall, Can’t tell)? Identify here the step of the process immediately following generation. Thisdoes not refer, necessarily, to the next item on the meeting agenda. Possibilities: narrow list,evaluate ideas, divide task among members, decide how to collect data, begin data collec-tion.

5. List and describe any technical problems and how the group responds. Technical prob-lems include any problems the group has with making the technology you identified inquestion 1 work as intended. Refer only to the technology used to record ideas. Do notinclude problems with other technologies (e.g., photocopiers, laptop computers, etc.) orwith technologies used for other purposes (e.g., to record agenda). Include here problemsboth with the technology itself and with the use of the technology (e.g., remembering howto use SAMM, recorder on flipchart has poor handwriting, etc.).

6. Sketch a general description of the task. Indicate here what seem to be the major stepsthe group follows. Descriptions of each step should indicate substance rather than simply ageneral category.

7. Sketch a map of the group.

Activity Elements

Ideation. Ideation is the public presentation of item for inclusion on a shared list. Collect:Who presented the idea;Content of the idea itself;When the idea was recorded in relation to when stated;Who recorded the idea;Whether the idea was linked to any other;Whether the idea was presented anonymously; andWhether the idea was later rejected within IG episode.


Criticism. Code as criticism any comment that implies the rejection of an idea as a legiti-mate or appropriate member of the list of ideas being generated. Two positions are present:one supporting the idea and one opposing it. During criticism periods, group discussionfocuses on whether or not to reject the idea. Criticisms are directed at ideas. Criticisms di-rected at people, technology, or the organization should be coded as tangents. Criticismneed not be negative or hostile in tone. A criticism may even support the idea itself, butreject it as acceptable for the list being generated. A criticism may occur as a single commentthat is not responded to. Collect:

Which idea was being criticized;Content of the criticism;How other members responded to the criticism;Whether the criticism resulted in rejection of the idea; andWho engaged in the criticism.Elaboration. Elaborations are any noncritical discussion of an idea that occurs immedi-

ately after the idea is presented. Any group member may take part in the elaboration, in-cluding the member who presented the idea. Elaboration does not require any response. So,for example, a period of elaboration may involve only one group member. Elaboration maybe of two types: clarification or rephrasing. Clarification explains; it implies a need to dis-cuss and understand an idea rather than a need to defend one. Members reserve judgmenton the idea, or some aspect of the idea, during this period. Thus, clarification occurs beforethe group has decided firmly whether the idea is an appropriate addition to the list. Re-phrasing occurs when the initial form or content of an idea is reworked substantially beforeit appears on the group’s final list. Rephrasing implies partial rejection of an idea in eitherits wording or its substance, but does not reject the essence or the kernel of the idea. Collect:

Which idea was being elaborated;Content of the elaboration;Whether the elaboration was clarification or rephrasing; andWho engaged in the elaboration.Tangent. Tangents are any discussion period 10 seconds or longer that moves two or

more group members away from beginning or continuing a period of idea-generation. Peri-ods may be co-coded as tangent and something else (e.g., direction, idea-generation, orsome other tangent). Collect:

Content of the tangent;The tangent type (e.g., task discussion, anecdotes, physical distractions);How the tangent ended;Whether the tangent included laughter;Whether the tangent interrupted the presentation of an idea; andWho engaged in the tangent.Direction. A direction is any statement intended to influence the action or response of

other members of the group. It is directed by one or more persons toward others. A direc-tion is complete in that it is reasonable to expect a group member to base specific action onit alone. Collect:

Content of the direction;Purpose of the direction;Group response to the direction; andWhether direction was solicited.


NOTES

1. Studies by Sutton and Hargedon (1996) and Paulus, Larey, and Ortega (1995) explorefunctions of brainstorming procedures other than idea-generation. These studies suggestthat brainstorming can be a participatory ritual that unifies members around an enjoyabletask and that brainstormers perceive each other favorably, regardless of the actual ideasgenerated in the sessions. However, whatever other functions brainstorming may have, itsimmediate purpose is to help the group perform its task.

2. One way to assist or encourage a certain procedure is through use of computerizedsystems designed to aid groups in making decisions (Dickson, Poole, & De Sanctis, 1992;McLeod, 1996). Typically, these group decision support systems (GDSSs) enable each mem-ber of a group to enter ideas at a terminal and then display the complete list of ideas on aprojector screen (McLeod, 1996). One of the most widely studied applications is ElectronicBrainstorming, a component of the GroupSystems GDSS (Dennis, Heminger, Nunamaker,& Vogel, 1990). Other studies have also analyzed systems that are based more closely on theOsborne procedure (Sosik, Avolio, & Kahai, 1998).

3. So, for example, in a five-person group there would be equal contributions to a par-ticular activity if each member engaged in 20% (.20) of that activity. Using a Euclidean mea-sure, this would produce a distance index of 0: (.20-.20)2 + (.20-.20)2 + (.20-.20)2 + (.20-.20)2+ (.20-.20)2 = 0.0. If the activities were distributed among the five members as 50%, 30%,20%, 0%, and 0%, the calculation would yield an index of 0.42: (.50-.20)2 + (.30-.20)2 + (.20-.20)2 + (.0-.20)2 + (.0-.20)2 = 0.42. The measure is sensitive to differences in group size. There-fore, we divided each resulting index by the maximum possible proportion of other activi-ties that a member could assume for that size group (1-1/N), standardizing the index sothat perfect equality would be represented by a score of 0.00 and perfect inequality by 1.00.

4. Studies were selected for comparison if they were well-known and widely cited. Wealso sought to represent an array of idea-generation conditions, including face-to-face inter-acting groups, nominal groups, and computer supported groups.

REFERENCES

Arnold, J. E. (1962). Useful creative techniques. In S. J. Parnes & H. F. Harding (Eds.), Asourcebook for creative thinking (pp. 251–268). New York: Charles Scribner’s Sons.

Bormann, E. G. (1985). Symbolic convergence theory: A communication formulation. Jour-nal of Communication, 35, 128–138.

Bouchard, T. J. (1972). Training, motivation and personality as determinants of the effective-ness of brainstorming groups and individuals. Journal of Applied Psychology, 56, 324.

Browning, L. D., & Beyer, J. M. (1998). The structuring of shared voluntary standards in theU.S. Semiconductor industry: Communicating to reach agreement. Communication Mono-graphs, 64, 1–25.

Daily B., Whatley A., Ash S. R., & Steiner R.L. (1996). The effects of a group decision supportsystem on culturally diverse and culturally homogeneous group decision making. Infor-mation and Management, 30, 281–289.

Delbecq, A., Van de Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program plan-ning: A guide to nominal and Delphi processes. Glenview, IL: Scott, Foresman.

Dennis, A. R., Heminger, A. R., Nunamaker, J. F., Jr., & Vogel, D. (1990). Bringing automatedsupport to large groups: The Burr-Brown experience. Information and Management, 18,111–121.

DeSanctis, G., & Poole, M. S. (1994). Capturing the complexity in advanced technology use:Adaptive structuration theory. Organization Science, 5, 121–147.


Dickson, G. W., Poole, M. S., & DeSanctis, G. (1992). An overview of the Minnesota GDSSresearch project and the SAMM system. In R. Bostrom, R. T. Watson, & S. Kinney (Eds.),Computer augmented teamwork: A guided tour (pp. 163–180). New York: Van NostrandRinehold.

Diehl, M., & Stroebe, W. (1991). Productivity loss in idea-generation groups: Tracking downthe blocking effect. Journal of Personality and Social Psychology, 61, 392–403.

Ellis, D. G., & Fisher, B. A. (1994). Small group decision-making: Communication and the groupprocess (4th ed.). New York: McGraw-Hill

Gallupe, B., Bastianutti, L., & Cooper, W. H. (1991). Unblocking brainstorms. Journal of Ap-plied Social Psychology, 76, 137–142.

Gallupe, R. B., Cooper, W. H., Grise, M., & Bastianutti, L.M. (1994). Blocking electronic brain-storms. Journal of Applied Psychology, 79, 77–86.

George, J. F., Easton, G. K., Nunamaker, J. F., Jr., & Northcraft, G. B. (1990). A study of col-laborative group work with and without computer-based support. Information SystemsResearch, 1, 394–415.

Geschka, H. (1993). The development and assessment of creative thinking techniques: AGerman perspective. In S.G. Isaksen, M.C. Murdock, R.L. Firestein, & D.J. Treffinger,(Eds.), Nurturing and developing creativity: The emergence of a discipline. (pp. 215–236).Norwood, NJ: Ablex.

Geschka, H., Schaude, G. R., & Schlicksupp, H. (1975). Modern techniques for solving prob-lems. In M. M. Baldwin (Ed.), Portraits of complexity: Applications of systems methodologiesto societal problems (pp. 1–7). Columbus, OH: Battelle Memorial Institute.

Gryskiewicz, S. S. (1987). Predictable creativity. In S. G. Isaksen (Ed.), Frontiers of creativityresearch: Beyond the basics (pp. 305–313). New York: Bearly Limited.

Hollingshead, A. B., McGrath, J. E., & O’Connor, K. M. (1993). Group task performance andcommunication technology: A longitudinal study of computer mediated vs. face-to-facework groups. Small Group Research, 24, 307–333.

Jablin, F. M. (1981). Cultivating imagination: Factors that enhance and inhibit creativity inbrainstorming groups. Human Communication Research, 7, 245–258.

Jablin F. M., Seibold D. R., & Sorenson R. L. (1977). Potential inhibitory effects of groupparticipation on brainstorming performance. Central States Speech Journal, 28, 113–121.

Jackson, M. H., Poole, M. S., & Kuhn, T. (2002). The social construction of technology andstudies of the workplace. In L. Lievrouw & S. Livingstone (Eds.), The handbook of newmedia (pp. 236–253). London: Sage.

Jarboe, S. (1996). Procedures for enhancing group decision making. In R. Y. Hirokawa & M.S. Poole (Eds.), Communication and group decision-making (2nd ed., pp. 345–383). Thou-sand Oaks, CA: Sage.

Juran, J. M., & Gryna, F. M., Jr. (1980). Quality planning and analysis: From product developmentthrough use. New York : McGraw-Hill.

Mason, J. G. (1962). Suggestions for brainstorming technical and research problems. In S. J.Parnes, & H. F. Harding (Eds.), A sourcebook for creative thinking (pp. 291–295). New York:Charles Scribner’s Sons.

McGrath, J. E. (1984). Groups: Interaction and performance. Englewood Cliffs, NJ: PrenticeHall.

McKinlay, A., Procter, R., & Dunnet, A. (1999). An investigation of social loafing and socialcompensation in computer-supported cooperative work. In C. Simone & S. Weisband(Eds.) Proceedings of Group’99, the International Conference on Supporting Group Work, 249–257. ACM Press.

McLeod, P. (1996). New communication technologies for group decision making: Towardan integrative framework. In R. Y. Hirokawa & M. S. Poole (Eds.), Communication andgroup decision-making (2nd ed., pp. 426–461). Thousand Oaks, CA: Sage.

Nutt, P. C. (1984). Planning methods for health and related organizations. New York: John Wiley.Osborne, A. (1957). Applied imagination. New York: Scribner.Paulus, P. B., Larey, T. S., & Ortega, A. H. (1995). Performance and perception of brainstormers

in an organizational setting. Basic and Applied Social Psychology, 17, 249–265.


Pinsonneault, A., Barki, H., Gallupe, B., & Hoppen, N. (1999). Electronic brainstorming: Theillusion of productivity. Information Systems Research, 10, 110–133.

Poole, M. S., & Baldwin, C. L. (1996). Developmental processes in group decision-making.In R. Y. Hirokawa & M. S. Poole (Eds.), Communication and group decision-making (2nded., pp. 215–241). Thousand Oaks, CA: Sage.

Poole, M. S., & DeSanctis, G. (1990). Understanding the use of group decision support sys-tems: The theory of adaptive structuration. In J. Fulk & C. Steinfield (Eds.), Organizationsand communication technology (pp. 175–195). Newbury Park, CA: Sage.

Poole, M. S. & Roth, J. (1989). Decision development in small groups IV: A typology ofdecision paths. Human Communication Research, 15, 323–356.

Poole, M. S., Seibold, D. R., & McPhee, R. D. (1985). Group decision-making as astructurational process. Quarterly Journal of Speech, 71, 74-l02.

Poole, M. S., Siebold, D. R., & McPhee, R. D. (1996). The structuration of group decisions. InR. Y. Hirokawa & M. S. Poole (Eds.), Communication and group decision-making (2nd ed.,pp. 114–146). Thousand Oaks, CA: Sage.

Sosik J. J., Avolio B. J., & Kahai, S. S. (1998). Computer-supported work group potency andeffectiveness: The role of transformational leadership, anonymity, and task interdepen-dence. Computers and Human Behavior, 14, 491–511.

Steiner, I. D. (1972). Group process and productivity. New York: Academic Press.Straus, S. G., & McGrath, J. E. (1994). Does the medium matter? The interaction of task type

and technology on group performance and member reactions. Journal of Applied Psychol-ogy 79, 87–97.

Sunwolf, & Seibold, D. R. (1999). The impact of formal procedures on group processes,members, and task outcomes. In L. R. Frey, D. S. Gouran, & M. S. Poole (Eds.) Handbookof group communication theory and research (pp. 395–431). Thousand Oaks, CA: Sage.

Sutton, R. I., & Hargedon, A. (1996). Brainstorming groups in context: Effectiveness in aproduct design firm. Administrative Science Quarterly, 41, 685–718.

Valacich, J. S., Paranka, D., George, J. F., & Nunamaker, J.F. (1993). Communicationconcurrency and the new media: A new dimension for media richness. CommunicationResearch, 20, 249–276.

Idea-Generation in Naturally Occurring Contexts: Complex Appropriation of a Simple Group Procedure

Documents

Transcript of Idea-Generation in Naturally Occurring Contexts: Complex Appropriation of a Simple Group Procedure