0

TITLE:

Cohorts and Relatedness: Self-Determination Theory As an Explanation of How Learning

Communities Affect Educational Outcomes

AUTHORS:

Martine Robinson Beachboard1

John C. Beachboard2

Wenling Li3

Stephen R. Adkison4

Author for correspondence

Dr. Martine Robinson Beachboard

Department of Mass Communication

Idaho State University

Campus Stop 8242

921 S. 8th

Ave.

Pocatello, ID 83209

USA

Telephone: 208 282-5395

Fax: 208 282-2258

e-mail: beacmart@isu.edu

Suggested Running Head:

Relatedness and Learning Communities in Higher Education

1 Associate Professor, Department of Mass Communication, Idaho State University, Pocatello,

ID, USA

2 Professor, Department of Computer Information Systems, College of Business, Idaho State

University, Pocatello, ID, USA

3 Professor and Director of the Ph.D Program, College of Education, TUI University, Cypress,

CA, USA

4 Provost and Senior Vice President for Academic Affairs, Professor of English, Eastern Oregon

University, La Grande, OR, USA

1

Cohorts and Relatedness: Self-Determination Theory As an Explanation of How Learning

Communities Affect Educational Outcomes

Abstract This study examines whether feelings of relatedness constitute a substantial means

by which learning communities (cohorts) improve learning outcomes in higher education. It

applies Ryan and Deci’s Self-Determination Theory to an analysis of the National Survey of

Student Engagement. The SDT hypothesizes that environments that support perceptions of

social relatedness improve motivation, thereby positively influencing learning behavior. The

authors propose that participation in cohort programs constitutes such an environment.

Measuring student perceptions of the contributions of their institutions, the study found increased

relatedness to peers and faculty and increased higher order thinking assignments (a control

variable included in the research model) to be substantial predictors of educational outcomes

relevant to literacy, critical thinking, and, especially, job preparation. The researchers suggest

that institutions will want to ensure that their learning community designs enhance student

feelings of relatedness.

Key Words Self-Determination Theory, National Survey of Student Engagement, NSSE,

relatedness, cohorts, learning communities.

2

Employers are complaining about college graduates’ inability to solve problems on their own,

write effectively, or work well in teams. The concerns are not unique to a few scattered

institutions. Many educators and government- and industry-sponsored consortia have questioned

whether U.S. higher education is adequately preparing students to meet 21st Century challenges

(Bennis & O'Toole, 2005; NLCLEAP, 2007; Department of Education, 2006; Education

Consortium, 2006; Ghoshal, 2005; Kutner et al., 2007). Higher education institutions are

re-examining their educational practices and seeking to demonstrate that they are indeed

providing students with the knowledge and skills required to survive and thrive in society (Carle,

Jaffee, Vaughan & Eder, 2009; Education Consortium, 2006; Klein, Kuh, Chun, Hamilton &

Shavelson, 2005). They have proposed a variety of strategies to improve student learning.

Our interest in one of those strategies, establishing formal learning communities (or

educational cohorts), was piqued when the college of business at our university considered

establishing a mandatory cohort program. Primary objectives for that program were: to improve

students’ critical thinking skills, overall learning, and ability to work in teams; to facilitate better

integration of content across the curriculum; and to enhance student retention. The observed

outcomes resulting from this initiative were mixed. While the literature on learning communities

and cohorts (we use these terms interchangeably in this article) is largely positive, notable

exceptions occur. Quite negative results also have been reported (Jaffee, 2007; Saltiel & Russo,

2001; Seifert & Mandzuk, 2006). We wanted to better understand what makes a learning

community work well and how negative outcomes might be avoided.

Researchers have defined learning communities in various ways (Zhao & Kuh, 2004;

Sapon-Shevin & Chandler-Olcott, 2001; Wesson, Holman, Holman & Cox, 1996). Formal

learning communities may or may not feature accelerated class formats and collaborative [or]

3

team learning (Saltiel & Russo, 2001) or a residential component (Inkelas & Weisman, 2003;

Zhao & Kuh, 2004), but all initiatives worthy of the term include enhanced “interaction of

students [fostered] by the intensity and exclusivity [of a] closed membership and impermeable

boundary” (Saltiel & Russo, 2001, p. 3).

Learning communities are thought to be particularly well suited to helping students

improve their critical thinking and communication abilities (Inkelas & Weisman, 2003;

MacGregor, 1991; Norris & Barnett, 1994; Pastors, 2006; Saltiel & Russo, 2001; Schmuck,

1988). Many studies have provided evidence indicating that participants had better “learning

outcomes” (Zhao & Kuh, 2004), “learned better” than nonparticipants (Tinto, Goodsell & Russo,

1994), took more responsibility for their own learning, got higher grades, and demonstrated

greater persistence or retention (Inkelas & Weisman, 2003; Stassen, 2003; Taylor, 2003; Tinto,

Goodsell & Russo, 1993; Tinto et al., 1994).

However, other studies identified cases where cohorts were not actually conducive to

learning (Jaffee, 2007; Saltiel & Russo, 2001; Seifert & Mandzuk, 2006). Pike (2000, p. 14)

suggests that: “…learning communities, in and of themselves, do not cause a student to learn,

make good grades or be retained.” In a critique of current research, he argues that “effects of

learning communities are indirect” and that it’s “extremely difficult to identify these indirect

effects” without “an explicit theory of student learning” guiding the assessment.

To further our understanding of learning community effects, we have incorporated into

our analysis the Self-Determination Theory concept of relatedness, which Ryan & Deci (2000)

associate with belongingness or connectedness. We propose that cohort participation enhances

student feelings of relatedness, which leads to improved student motivation and educational

4

outcomes: specifically, students’ ability to communicate effectively, think critically and

analytically, and generally be better prepared to enter the workforce.

Accordingly, the following purposes motivated this research. We sought stronger

empirical evidence about whether the anticipated student benefits of cohort participation merit

the institutional costs, scheduling constraints, and instructor efforts required to implement and

administer such programs. Our study also was designed to make a theoretical contribution by

testing whether cohort participation can facilitate greater feelings of relatedness and whether

higher levels of relatedness contribute to improved learning outcomes. That is, the study seeks

to explain some of the indirect effects as suggested by Pike.

Previous Research

Learning Community Effects

Previous studies investigating cohort influence on student performance have reported mixed

results ranging from very positive to very negative (Dyson & Hanley, 2002; Jaffee, 2007; Seifert

& Mandzuk, 2006; Shapiro & Levine, 1999; Tinto et al., 1994). Advocates of cohort

programming say the arrangement promotes exchange of ideas, critical feedback, and

effectiveness of learners’ efforts (Saltiel & Russo, 2001). Zhao and Kuh (2004, p. 127) reported

that learning community participation was associated with student gains in “practical

competence and general education.” Norris and Barnett (1994, p. 1) noted reports of “enhanced

knowledge” in cohort participants. Students in some learning communities felt they were

gaining improved skills in reading, writing, decision-making, and oral presentation (Saltiel &

Russo, 2001). The mechanism for some of the claimed improvement in student outcomes was

identified as the fostering of a safe environment to “build a network of peers” (Tinto et al., 1993,

5

p. 18). Participants felt “a sense of security that allowed them to speak freely without fear of

peer criticism” (Maher, 2005, p. 204).

Reports of negative cohort experiences abound as well (Connor & Killmer, 2001; Drago-

Severson et al., 2001; Dyson & Hanley, 2002; Mather & Hanley, 1999; Reynolds & Sitharaman,

2000; Ross, Stafford, Church-Pupke & Bondy, 2006; Wheelan & Lisk, 2000). Several

researchers noted excessive socializing, misconduct, disruptive and rebellious behavior, and the

formation of cliques (Eder & Enke, 1991; Jaffee, 2007; McFarland, 2001; Milner, 2004;

Ridgeway, 1983; Saltiel, 1994; Saltiel & Russo, 2001). Jaffee (2007, pp. 65-67) found behaviors

“that may hinder student learning, student development, and faculty-student relations.” He also

noted that cohorts’ “behavioral conformity” can give rise to “the groupthink phenomenon that

produces mutually reinforcing views and perspectives” about an assignment, reading, or

evaluation. Cohort cohesion or bonding can backfire: “When the power of the group is strong,

they can weaken professors by challenging their authority and knowledge base in such a way that

at one institution an instructor was left cowering in a corner” (Saltiel & Russo, 2001, p. 61).

Saltiel & Russo (2001, p. 59) also warned of “intellectual inbreeding” in which students cease to

be surprised or inspired by one another’s remarks. The familiarity bred in cohorts can result in a

“comfort zone [that signals] an intellectual plateau at which students are no longer challenged to

grow but… maintain the status quo” (Maher, 2005, p. 208).

In short, the culture of cohorts can be very positive or negative (Kelly & Dietrich, 1995;

Radencich et al., 1998; Sapon-Shevin & Chandler-Olcott, 2001). The contradictory findings

attest to the complexity of learning communities and the challenges that researchers face in

assessing them whether due to sampling, measurement, or definition issues. Why do some

learning communities perform so well while others appear to be so dysfunctional? Consistent

6

with Pike’s critique of the literature, we find that most research does not report on intervening

variables that could help explain cohort successes and failures. We consulted two distinct

research streams to identify potentially helpful intervening variables.

Self-Determination Theory: A Lens for Examining Learning Community Effects

This study applies Self-Determination Theory (SDT) to examine whether cohort-induced

changes in the learning environment are associated with improved learning outcomes (Ryan &

Deci, 2000; Ryan & Deci, 2002; Ryan, Connell & Deci, 1985). The SDT is a theory of

motivation. Ryan and Deci (2000, p. 69) describe motivation as the “energy, direction,

persistence… aspects of activation and intention” that address the why of human behavior.

Motivation is vital: if students do not want to learn, little learning is likely to take place

“because learning is an active process requiring conscious and deliberate effort” (Stipek, 1988,

p. ix). Numerous studies have confirmed that intrinsic or more internalized forms of motivation

are associated with increased interest, engagement, effort, learning, and satisfaction with

education (Boyd, 2002; Connell & Wellborn, 1991; Grolnick & Ryan, 1987; Hayamizu, 1997;

Kowal & Fortier, 1999; Miserandino, 1996; Ryan & Deci, 2000; Standage, Duda & Ntoumanis,

2003; Vallerand & Bissonnette, 1992; Vansteenkiste, Lense & Deci, 2006).

Deci and Ryan (1985) identified three basic needs conducive to the development of more

highly internalized motivation. These are: autonomy, competence, and relatedness. Autonomy

is an internal perceived locus of causality (deCharms, 1968; Ryan & Deci, 2000). Deci and

Ryan posit that choice and autonomy enhance intrinsic motivation. For example, if a student

does his homework because his parents insist on it, he does so out of compliance and with a low

sense of autonomy. However, if a student does her homework believing that it will improve her

performance in a desired occupation, she is still extrinsically motivated but has given the activity

7

“personal endorsement,” a higher level of autonomy (Ryan & Deci, 2000, p. 71). Competence is

conceptualized as a sense of self-efficacy (Harter, 1992; Ryan & Deci, 2000). It is vital in the

motivating scenario because people adopt activities that make them feel their actions affect

outcomes (Ryan & Deci, 2000). Relatedness is described as “the need to feel belongingness and

connectedness with others” (Ryan & Deci, 2000, pp. 68-69,73). [See also: White (1959), Harter

(1992), deCharms (1968), Deci (1975), Baumeister and Leary (1995), and Reis (1994).]

Researchers have noted that learning environments promoting a sense of relatedness to teachers,

parents, and peers can strengthen motivation and have a positive effect on school outcomes

(Chen & Jang, 2010; Ryan & Deci, 2000; Ryan & Grolnick, 1986; Ryan, Stiller & Lynch, 1994).

Feelings of relatedness, measured in terms of “school climate” and teacher-student relationships,

have been linked to outcomes including self-efficacy, engagement, interest in school, higher

grades, and retention (Furrer & Skinner, 2003; Inkelas & Weisman, 2003; Inkelas, Daver, Vogt

& Leonard, 2007). The similarity between the relatedness construct employed in this study and

sense of belongingness, which has been examined as a predictor of student persistence, further

signals the relevance of the relatedness construct in education research (Bean, 1980; Bean &

Metzner, 1985; Hausmann, Ye, Schofield & Woods, 2009; Tinto, 1987).

Given the explicit emphasis on social learning and developing a sense of community that

cohorts are typically associated with, we propose that the relatedness construct may be a

critically important intervening variable to consider when attempting to understand the effect of

cohort participation on student learning. That is, we propose that cohorts work, at least in part,

by providing learning environments that increase students’ feelings of relatedness which, in turn,

influence academic performance.

8

Critical and Reflective Thinking

Given our particular interest in the development of communication, critical thinking, and

problem-solving skills, we thought it important to include student exposure to higher order

thinking assignments as an exogenous/control variable to help isolate effects that can be uniquely

attributed to cohort participation. Accordingly, we examined the literature on literacy and

critical thinking to learn how these concepts have been conceptualized and to identify specific

educational strategies that have been used to promote their development. The critical thinking

literature emphasizes the cognitive processes associated with applying information and

recognizing the uncertainty inherent in making decisions. Critical thinking is concerned with the

ability of an individual to evaluate arguments and evidence, construct rationales for beliefs, and

examine his or her own reasoning (Bruning, 1994; King & Kitchener, 1994). While multiple

perspectives exist, the predominant perspective in the literature conceives of critical thinking as

reflective judgment characterized by an inquiry or problem-solving process where a

demonstrably correct solution cannot be identified (King & Kitchener, 1994; King & Kitchener,

2004; Wolcott, 2006). Literacy literature also emphasizes the need to carefully formulate a

problem or question to be answered; identify, evaluate, and organize information for practical

application; and integrate new information into existing knowledge (Eisenberg, Lowe & Spitzer,

2004; King & Kitchener, 2004).

Research Questions

This study draws from a large sample of students participating in a large number of learning

communities to rigorously assess whether cohort participation consistently results in improved

learning outcomes (e.g., ability to communicate, solve complex problems, and work well with

9

others) and to examine whether feelings of relatedness help explain observed cohort effects.

The study addresses three research questions:

(1) Does student participation in formal learning communities (cohorts) lead to higher levels

of academic development and job preparation?

(2) Do higher degrees of relatedness to faculty and peers predict higher levels of student

academic development and job preparation?

(3) Does relatedness mediate the effect of learning community (cohort) participation on

student academic development and job preparation?

Methods

Data and Sample

The data source for this study was the 2005 National Survey of Student Engagement (NSSE,

2005). The selection of the NSSE as a data source was driven by our desire to access students

participating in a large number of cohort initiatives. This criterion is based on our concern that

the conflicting findings in the published research may result in part from the relatively small

sample of cohorts examined in individual studies.

The NSSE instrument relies on student self-reports and was designed to satisfy attendant

validity conditions (Bradburn & Sudman, 1988; NSSE, 2007; Pace, 1984; Pike, Schroeder &

Berry, 1995). The Indiana University Center for Postsecondary Research (IUCPR) restricted our

sample to 2,000 records randomly drawn from 130 of the 529 four-year colleges and universities

participating in the 2005 NSSE. We requested a stratified sample with equal numbers of males

and females, and cohort participants and nonparticipants. The IUCPR randomly drew a sample

which met the following additional criteria: full-time student status; seniors only (they had had

more opportunity to participate in a learning community experience and were better positioned to

10

assess institutional contributions to their academic development and job preparation); non-

international (to reduce the confounding influence of cultural differences and to eliminate small

cell sizes); and institutions classified by the Carnegie Foundation (at the time the data were

collected) as public, master’s degree-granting, and doctoral/research-intensive universities. We

excluded private institutions because they may be more likely to provide learning community-

type environments even without establishing formal cohorts. We additionally excluded research-

extensive institutions because of their more selective enrollment policies and because they tend

to be more narrowly focused on research output than mid-tier universities (Hausmann et al.,

2009). Our goal was to achieve a homogeneous sample relevant to the educational experiences

of most undergraduate students in the United States (NSSE, 2005), minimizing variance within

groups to improve our focus on between-group differences, as suggested by Kerlinger (1986).

Research Model

Based on our review of the literature, the research model at Figure 1 depicts the influence of

learning community participation on student academic development and job preparation as being

mediated by feelings of relatedness. A direct path between learning community participation and

the response variables also is included as we were not prepared to assume that the influence was

fully mediated. The research model controls for exogenous variables found by previous

researchers to influence educational outcomes [e.g., Pascarella and Terenzini (1991), Roksa

(2010), and Zhao and Kuh (2004)]. These control variables include standard demographics and

SAT scores (or ACT scores converted to SAT equivalents) to control for a variety of

demographic attributes and student characteristics. Our original model also controlled for

parental education, but preliminary analysis showed it to correlate strongly with SAT.

Consequently, we excluded the non-essential variable to minimize artificial inflation of the R2

11

and improve model parsimony. Additional controls were student exposure to higher order

thinking assignments and participation in a variety of alternative activities held to be

educationally enriching, as these constructs have also been shown to correlate with the selected

response variables (Ennis, 1985; Glaser, 1985; King & Kitchener, 1994; Lipman, 1988; Perry,

1970; Zhao & Kuh, 2004).

------------------------

Place Figure 1 about here

------------------------

Study Measures

The NSSE instrument provided an extremely rich data source for this study. As have other

researchers, we adapted several previously reported scales to sharpen the conceptual focus of

constructs relevant to the needs of our study (Carle et al., 2009; Carini, Kuh & Klein, 2006;

Gordon, Ludlum & Hoey, 2008; Kuh, 2001; Pike, 2006a; Pike, 2006b). We examined the

psychometric properties of multi-item scales to ensure the reliability of selected measures.

Table 1 provides the definitions and composition of study measures.

------------------------

Place Table 1 about here

------------------------

An advantage we had over the previous learning community study conducted by Zhao

and Kuh (2004) is that the NSSE instrument has been modified to clearly differentiate between

intention to participate in an enriching learning activity and actual participation. This change in

the survey instrument permitted us to clearly dichotomize participation versus nonparticipation

in cohorts and other enrichment activities. Three enrichment activities were included as controls

because they “may well have similar positive effects” (Zhao & Kuh, 2004, p. 132). We were

12

able to derive a useful measure of relatedness by combining six NSSE items previously used in

formulating scales to assess student interactions with faculty and supportive campus

environments. Our higher order thinking construct draws on six NSSE questions addressing

higher levels of Bloom’s classic (1956) Taxonomy of Educational Objectives.

Analysis

We examined the data using descriptive statistics to check for outliers and non-normal

distributions. We also performed bivariate analysis using Student’s t-test of independent group

means to examine whether and to what extent relationships existed between cohort participation

and other study variables. The primary analyses employed simple linear regression and block-

entry regression. In block-entry regression models, the researcher determines the entry order of

control, independent and mediating variables, and can more easily assess their unique impact on

the research model (Meyers, Gamst & Guarino, 2006). We performed statistical tests checking

for violation of regression assumptions pertaining to data normality, homoscedasticity, and

multi-collinearity. Residual plots were examined to check whether the assumption of linearity

was violated. We tested a simple linear regression model to ascertain the effect size of cohort

participation on the dependent variables without including any potentially confounding variables.

If we had found no significant effects associated with this simplest of statistical models, further

analysis would not have been warranted.

Given the pragmatic objectives of this research, we calculated Y-standardized effect sizes

where appropriate. We considered an effect size of less than .10 of a standard deviation in a

response variable to be too small to warrant consideration in decision-making (Alexander &

Pallas, 1985; Rosenthal & Rosnow, 1991; Zhao & Kuh, 2004). Only after confirming that the

anticipated relationship between cohort participation and the response variables existed did we

13

proceed to block-entry regression models to rigorously assess the unique impact of cohort

participation in the presence of the identified control variables.

To examine the degree to which relatedness mediates the effects of cohort participation

on the two outcome variables, we used a procedure outlined by Baron and Kenny (1986).

Results

We began our analysis by examining frequencies and percentages for categorical variables,

resulting in the working sample depicted at Table 2. Due to extremely small cell sizes, we

eliminated age outliers and reduced the number of ethnic categories. These decisions resulted in

a working sample of 1,852 potentially usable records. In the interest of preserving all usable

records, we used listwise treatment of missing values within each statistical model. The

distribution of missing values reduced the number of records and caused slight variations (1,845

when regressing on job preparation and 1,846 on academic development).

------------------------

Place Table 2 about here

------------------------

We next ran descriptive statistics on all scales to check for normality, skew, and ceiling or floor

effects and noted no adverse conditions. We then performed bivariate analysis to determine

whether linear relationships between key variables could be identified. Student’s t-tests revealed

statistically significant differences between cohort participants and nonparticipants on three

control variables: adjusted SAT, higher order thinking assignments, and enrichment activities

(see Table 3). Examination of residual plots did not identify nonlinear effects. For higher order

thinking we found a 1.29 difference and for enrichment activities a .38 difference (p < .001).

Similar to the findings observed by Zhao and Kuh (2004), the adjusted SAT scores of cohort

14

participants in our sample were lower than those of nonparticipants: in our case, a difference of

nearly 20 points (p < .01).

------------------------

Place Table 3 about here

------------------------

Findings Relevant to the Efficacy of Learning Community Participation (Research Question 1)

We began our substantive analysis by running two simple linear regression models, one for each

dependent variable. Cohort participation was the sole predictor variable used. The results were

statistically significant (p < .001) and met our criteria for substantial significance (see Table 4).

The Y-standardized effect sizes (.26 relative to academic development and .31 relative to job

preparation) substantially exceeded the minimum .10 effect size considered useful in terms of

decision-making (Alexander & Pallas, 1985; Rosenthal & Rosnow, 1991; Zhao & Kuh, 2004).

Cohort participation explained 1.8% of academic development variance and 2.4% of job

preparation variance.

------------------------

Place Table 4 about here

------------------------

We then ran block-entry regression models for both dependent variables (see Tables 5 and 6)

where control variables were entered in block 1, cohort participation in block 2, and the

relatedness construct in block 3 (Meyers et al., 2006). The focus of our findings relative to

research question 1 is limited to an examination of blocks 1 and 2; block 3 findings relate to

research questions 2 and 3.

In an analysis of the academic development outcome, the control variables of gender,

SAT, age, ethnicity, enrichment activities, and higher order thinking assignments entered at

15

block 1 were significant and in total accounted for 23 percent of observed variance in academic

development (R2 = .230, p < .001), as shown in Table 5. Importantly, in the presence of all of

these control variables, the entry of cohort participation into the regression at block 2 was not

statistically significant (compare with simple effects reported in Table 3).

The higher order thinking assignments variable is a significant predictor of academic

development (β = 0.442, p < .001) as shown in block 2 of Table 5. Given the relationship

between HOT and cohort participation previously identified (see Table 3), it appears that the

inclusion of this control variable suppressed the influence of cohort on academic development.

In an examination of the job preparation outcome, the control variables at block 1

accounted for 18.5 percent of observed variance (see Table 6). The entry of cohort participation

into the regression analysis at block 2 was statistically significant (β = .055, p < .05) but added

just 0.3 percent to variance explained (ΔR2 = .003). This is noteworthy but still less predictive

than higher order thinking (β = .374, p < .001) or enrichment activities (β = .073, p < .001).

------------------------

Place Table 5 about here

------------------------

------------------------

Place Table 6 about here

------------------------

The findings provide mixed results about the efficacy of cohort participation. The cohort

variable did not provide strong support for direct benefits of learning communities. The size of

the cohort participation regression coefficients was diminished for both response variables after

inclusion of control variables in the research models (compare the β values in Table 4 with

block 2 values in Tables 5 and 6). In fact, cohort participation was no longer statistically

significant as a predictor of academic development. And while cohort remained a statistically

16

significant predictor of job preparation, a Y-standardized effect size of .115 barely exceeded .10,

the minimum value considered relevant for decision-making purposes (Alexander & Pallas,

1985; Rosenthal & Rosnow, 1991; Zhao & Kuh, 2004, p. 123).

Support for SDT: The Influence of Relatedness on Academic Development and Job Preparation

(Research Question 2)

To address whether relatedness had a direct effect on academic development and job preparation,

we added the relatedness variable at the third block in the regression model. The results were

notable, with relatedness contributing 5.3 percent of variance explained for academic

development and 9.4 percent of variance explained for job preparation (see block 3 of Tables 5

and 6).

This analysis shows that relatedness demonstrated significant direct effect on both

response variables. Once relatedness was added to the models, cohort participation was no

longer significant on either response variable. Baron and Kenny (1986) advise that such a loss of

significance provides evidence that a mediation effect exists and the strongest demonstration of

mediation occurs when that path changes to zero. The proposition that relatedness mediates the

relationship between cohort participation and the outcome variables is formally tested in the

following section.

Relatedness As a Mediator of Cohort Participation Effects (Research Question 3)

To address whether relatedness has a mediating effect between cohort participation and

the educational outcomes, we performed the analysis prescribed by Baron and Kenny (1986).

This technique required the calculation of four equations (refer to Figure 2).

17

------------------------

Place Figure 2 about here

------------------------

The first equation was the regression of the explanatory variable, cohort participation, and

control variables on the response variables. A second equation was performed in which the

mediating variable, relatedness, was included in the model. The difference between regression

coefficients for cohort participation, with and without relatedness, was tested to determine if a

statistically significant mediating effect were present, using the formula:

To obtain the Sindirect term required to conduct this additional test of significance, two regression

equations were required: the DV was regressed on the mediating variable without the key IV

(represented by arrow “b” at Figure 2), and finally the mediating variable was regressed on the

key IV (arrow “a”). These equations provided the regression coefficients and standard error

terms required to populate the following equation for the standard error, Sindirect.

In the Sindirect equation above, b

2 represents the square of the unstandardized regression

coefficient as depicted in path “b,” and a2 represents path “a.” The “a” value is the same for both

calculations (a = .871). The “b” coefficients were calculated for each DV (b = .194 relative to

academic development and .2 relative to job preparation). The S terms represent the standard

error terms squared associated with regression coefficients in the paths. Again there is only one

error term for Sa (.154). Standard errors (Sb) relative to the DVs were .017 for academic

development and .013 for job preparation.

The results reported in Table 7 provide statistical evidence (p < .001) that relatedness

does mediate the effect of cohort participation on the response variables. These findings support

18

the proposition that cohort participation is associated with increased feelings of relatedness,

which in turn are associated with improved academic development and job preparation.

------------------------

Place Table 7 about here

------------------------

Summary of Findings

The direct effects of learning communities on the response variables, once control variables were

included, were modest. Cohort participation was statistically insignificant relative to academic

development and was statistically significant on job preparation with an effect size that barely

exceeded the minimum effect size of interest. A quick visual overview is offered at Table 8. It

shows the diminishing effects of cohort participation in the presence of other factors. The higher

order thinking control variable was a more significant predictor of the response variables than

was cohort participation. While outside of the scope of this study, the HOT-related finding is the

subject of a separate article we have published (Beachboard & Beachboard, 2010). Relatedness

proved to be the strongest predictor of the response variables, providing strong evidence in

support of the applicability of Self-Determination Theory. The finding that relatedness appeared

to mediate the relationship between cohort participation and educational outcomes suggests that

SDT may help explain benefits associated with learning communities.

------------------------

Place Table 8 about here

------------------------

Discussion

The primary motivation for this research was to help educators decide whether to implement

formal learning communities and identify important factors that could contribute to the design of

19

learning communities more effective at improving communication skills, critical thinking, and

job preparation. The study applied Self-Determination Theory, a theory of human motivation.

The findings (see Tables 5 and 6) provide compelling evidence in support of the SDT-

generated hypothesis concerning the relationship between students’ sense of relatedness and their

academic development and job preparation. Relatedness, a sense of “belongingness and

connectedness with others” (Ryan & Deci, 2000, p. 73), proved to be the single most influential

variable predicting student perceptions of their institutions’ contributions to their educational

development. This strongly supports the idea that motivation and, ultimately, student

performance co-vary with indicators that institutions are meeting student needs for feelings of

relatedness.

We examined relatedness as a mediator between cohort participation and the two

constructs employed to assess student educational outcomes. The findings (Table 7) support our

proposition that relatedness mediated the relationship between cohort participation and the

response variables, academic development and job preparation. These results have considerable

“face validity” given that cohorts generally are created with the intent of establishing an

environment facilitating social learning. The mediation effect also helps explain the diminished

significance of cohort participation as an explanatory variable (Baron & Kenny, 1986).

A reviewer of an earlier version of this paper commented on the absence of references to

belongingness, a construct that has been researched as a predictor of student persistence and

retention. The relatedness results reported here are consistent with previous research on

belongingness and student persistence (Bean, 1980; Bean & Metzner, 1985; Hausmann,

Schofield & Woods, 2007; Hausmann et al., 2009; Hurtado & Carter, 1997; Hurtado et al.,

2007).

20

The next issue is whether the study findings support institutional establishment of formal

learning communities or contribute to ideas for improving cohort design. The analysis of the

baseline model suggests that student participation in cohorts has positive effects on academic

development and job preparation – achieving both statistical and substantive significance as

determined by Y-standardized effect sizes. However, with the entry of the control variables,

cohort participation was no longer statistically significant relative to academic development.

And while it remained statistically significant regarding job preparation, the effect size barely

exceeded the minimum threshold for decision-making purposes. Thus, as Pike advised (2000),

cohorts in and of themselves would not seem to be called for in the absence of further

explanation. The current study provides some insight.

Two control variables that particularly dilute the cohort effect are enrichment activities

and higher order thinking. The bivariate analysis of independent means reported in Table 3

indicates that students participating in cohorts also experienced a larger number of educational

enrichment activities and had greater exposure to HOT assignments; since these two factors co-

vary with the response variables, their introduction into the model necessarily reduces the

observed cohort effects. This observation may help explain how cohorts work or why some

work better than others. It also raises the question of whether enrichment activities and higher

order thinking assignments could have been analyzed as mediating variables. We found no

logical basis for treating enrichment as a mediator, but proper treatment of the HOT variable is

less clear. Much cohort literature suggests that formal learning communities do provide an

environment conducive to the development of critical thinking ability (Jaffee, 2007; James,

Bruch & Jehangir, 2006; Norris & Barnett, 1994; Pastors, 2006; Saltiel & Russo, 2001; Tinto,

1988; Tinto, 1997). However, since we can easily imagine cohort-based curricula that do not

21

emphasize critical thinking and non-cohort settings that do, we would not endorse treating HOT

as a mediating variable. To confidently answer questions concerning the relationship between

cohort participation and higher order thinking assignments requires further research.

Finally, we comment on the disparate influence of cohort participation on the two

response variables (our finding that cohort participation lost its apparent effect on academic

development once control variables were taken into consideration). The academic development

construct addressed students’ writing effectiveness, critical thinking, and independent-learning

abilities. The job preparation variable addressed acquiring general work-related knowledge and

skills; solving complex, real-world problems; and working effectively with others. We have

addressed the interpretation that cohorts improve motivation by means of increased relatedness;

this is consistent with better learning or academic development. But we see no reason why

cohort participation would lead to greater exposure to actual content that is job-related. There is,

though, face validity to the proposition that learning communities would increase students’

ability to work with others and solve complex, real-world problems because of cohorts’ intense

interaction and closed memberships (Saltiel & Russo, 2001).

Implications for Practice

While the bulk of cohort research cited positive outcomes associated with formal learning

communities, we were concerned about the number of studies reporting notably negative

experiences. Prior to conducting our research, we found no convincing explanation for the

contradictory findings that might help institutions minimize their chances of experiencing

undesirable cohort outcomes. That is, the studies reporting negative results were primarily

descriptive, not explanatory; they did not measure intervening variables that might help

institutions predict or avoid problems.

22

We believe our study findings pertaining to the relatedness construct provide a useful perspective

on earlier contradictory reports in the literature. The group dynamics of particular cohorts are

likely a key to their success or failure. In educational contexts, it may be useful to distinguish

between relatedness among students and relatedness between students and faculty. Several of

the studies reporting negative outcomes described instances of group behavior of tightly bonded

students opposing instructor efforts and thwarting the educational objectives of the program

(Saltiel & Russo, 2001; Jaffee, 2007). On the other hand, Inkelas and Weisman (2003)

suggested that tight faculty mentoring relationships might positively influence students’

academic outcomes while detracting from their social transition to college. The learning

community experience at our university bears out this distinction. The primary motivations for

offering a cohort program were to help students improve their communication and problem-

solving skills, and to facilitate integration of interdisciplinary content. Specially selected faculty

and students participated in the “trial” cohorts, and the overall response from students was very

positive. The initial cohorts offered strong evidence of bonding among students and with

faculty. For example, one student noted in her evaluation of the program, “the [initiative] has

allowed the cohort to develop deeper ties with the faculty. Having deeper ties means a greater

opportunity for learning and growing” (Aytes, 2005).

However, the local cohort experience was not uniformly positive. Several faculty

members noted that students’ attitudes made class management more challenging, particularly

for less experienced instructors. The problem appeared to be that once significant bonding

occurred among students, or between students and cohort faculty, the students were less able or

willing to form productive bonds outside of the cohort. Some faculty members felt that the

quality of their classes was diminished by the development of “cliques” and that some cohort

23

students became disrespectful toward non-cohort members. As one local student commented,

“much like a large family, we have our inner squabbling, but in the big scheme of things we all

care very much for each other’s wellbeing and each other’s success. I have never felt the kind of

academic connection and social connection with people in my other college of business classes”

(Aytes, 2005). If cohort participants coalesce in opposition to faculty, there can be negative

consequences in terms of both faculty performance and student learning. The multiple available

examples of unproductive student behavior in the literature suggest that simply establishing

cohorts does not guarantee the development of healthy relationships on campus. Consequently,

we believe the issue of relatedness in the context of cohort design may be a key factor

determining the success or failure of particular cohorts. This merits further study.

The establishment of formal learning communities represents a manipulation of the

students’ social learning environment. To improve the probability of using social learning

communities successfully, institutions need to carefully monitor not only the internal dynamics

of cohorts but also the interaction of cohort students with the larger academic environment to

assure that they remain productive. We suggest they need to tend to the issue of relatedness.

Finally, we want to emphasize the explanatory power of higher order thinking assignments

relative to academic development and job preparation. While HOT was treated as an exogenous

variable in this study, the findings provide evidence that students with greater exposure to higher

order thinking assignments felt their institutions were making a greater contribution to their

educational development, whether they were in cohorts or not. Accordingly, we suggest that

institutions should encourage faculty to incorporate a variety of higher order thinking activities

in their course designs.

24

Limitations

This research shares a limitation with many other studies: lack of random selection and

assignment of study participants to treatment and non-treatment groups. Our analysis indicated

that some statistically significant differences existed between cohort participants and

nonparticipants. Without random assignment, there is no statistical control over what might be

relevant but unmeasured differences between learning community participants and non-

participants. Also, as noted by Zhao and Kuh in their 2004 study of learning communities,

researchers employing the NSSE data set are unable to distinguish type and timing of cohort

participation. Thus, we were unable to distinguish between stronger or weaker cohort designs,

potentially understating the effects of well designed learning community initiatives.

Furthermore, the NSSE data represent a snapshot in time, include a limited number of

questions possibly omitting some educationally relevant activities, and rely on student self-

reports from respondents who may be operating from different perspectives (Pascarella, 2001;

Pike, 2000; Zhao & Kuh, 2004). In particular, the study does not directly assess learning

outcomes, but students’ perceptions of the degree to which their institutions contributed to their

learning. However, we felt this perceptual measure was more relevant to answering our research

questions than more commonly used measures of student performance such as grades. The

questionnaire was designed to satisfy attendant validity conditions (Bradburn & Sudman, 1988;

NSSE, 2007; Pace, 1984; Pike et al., 1995), and an analysis of the statistical findings

demonstrated acceptable levels of content and predictive validity.

Conclusions

In summarizing his findings over multiple years of involvement with NSSE, Kuh wrote (2007,

pp. 8-9):

25

What one thing can we do to enhance student engagement and increase student

success? …Make it possible for every student to participate in at least two high

impact activities during their undergraduate program… left to their own devices,

many students and faculty members may not do these things. Educationally

effective institutions recognize this and create incentives to purposeful behaviors

towards these ends.

We agree with Kuh that there is no single educational panacea; an institution must provide a

range of high impact activities and an overall educational environment that facilitates student

motivation. Our research substantiates that learning community participation should be

considered among those high-impact activities. This is particularly the case if program

developers are interested in helping students sharpen their ability to work with others in solving

complex, job-related problems, because we found that cohorts influenced job preparation more

than the other outcomes. And within the framework of learning community establishment, we

again emphasize the particular aspect upon which we have been able to shed some additional

light: the fostering of academic relatedness between students and faculty.

References

Alexander, K., & Pallas, A. (1985). School Sector and Cognitive Performance: When Is a Little

a Little? Sociology of Education, 58(2), 115-128.

Beachboard, M. Robinson & Beachboard, J. C. (2010). Critical Thinking Pedagogy and Student

Perceptions of University Contributions to Their Academic Development. Informing

Science: The International Journal of an Emerging Transdiscipline, 13:53-71.

Aytes, K. (2005). Technology-Enhanced Core Curriculum Status Report. Pocatello, Idaho.

Baron, R. M., & Kenny, D. A. (1986). The Moderator-Mediator Variable Distinction in Social

Psychological Research: Conceptual, Strategic and Statistical Considerations. Journal of

Personality and Social Psychology, 51(6), 1173-1182.

Baumeister, R., & Leary, M. (1995). The Need to Belong: Desire for Interpersonal Attachments

As a Fundamental Human Motivation. Psychological Bulletin, 117, 497-529.

Bean, J. (1980). Dropouts and Turnover: The Synthesis and Test of a Causal Model of Student

Attrition. Research in Higher Education, 12, 155-187.

Bean, J., & Metzner, B. (1985). A Conceptual Model of Nontraditional Undergraduate Student

Attrition. Review of Educational Research, 55(4), 485-540.

Bennis, W. G., & O'Toole, J. (2005, May). How Business Schools Lost Their Way. Harvard

Business Review, pp. 96-104.

26

Bloom, B. S., & Krathwohl, D. R. (1956). Taxonomy of Educational Objectives: The

Classification of Educational Goals. In Handbook I: Cognitive Domain. New York:

Longmans, Green.

Boyd, F. B. (2002). Motivation to Continue: Enhancing Literacy Learning for Struggling

Readers and Writers. Reading & Writing Quarterly, 18, 257-277.

Bradburn, N., & Sudman, S. (1988). Polls and Surveys: Understanding What They Tell Us. San

Francisco: Jossey-Bass.

Bruning, R. (1994). The College Classroom from the Perspective of Cognitive Psychology. In K.

Prichard & R. Sawyer (Eds.), Handbook of College Teaching. Westport, CT: Greenwood

Press.

Carini, R. M., Kuh, G. D., & Klein, S. P. (2006, February). Student Engagement and Student

Learning: Testing the Linkages. Research in Higher Education, 47(1), 1-32.

Carle, A. C., Jaffee, D., Vaughan, N. W., & Eder, D. (2009, 30 June). Psychometric Properties of

Three New National Survey of Student Engagement Based Engagement Scales: An Item

Response Theory Analysis. Research in Higher Education, 50(8), 775-794.

Chen, K.-C., & Jang, S.-J. (2010, 11 February). Motivation in Online Learning: Testing a Model

of Self-Determination Theory. Computers in Human Behavior, 26, 741-752.

Connell, J., & Wellborn, J. (1991). Competence, Autonomy and Relatedness: A Motivational

Analysis of Self-System Processes. In M. Gunnar & L. Sroufe (Eds.), Minnesota

Symposium on Child Psychology (Vol. 22) (pp. 43-77). Hillsdale, NJ: Erlbaum.

Connor, K. R., & Killmer, N. (2001). Cohorts, Collaboration and Community: Does Contextual

Teacher Education Really Work? Action in Teacher Education, 23(3), 46-53.

deCharms, R. (1968). Personal Causation. New York: Academic Press.

Deci, E. L. (1975). Intrinsic Motivation. New York: Plenum Press.

Deci, E. L., & Ryan, R. M. (1985). Intrinsic Motivation and Self-Determination in Human

Behavior. New York: Plenum Press.

Department of Education. (2006). The Secretary of Education's Commission on the Future of

Higher Education. In A test of leadership: Charting the future of U.S. higher education.

Washington, DC: Education Publication Center.

Drago-Severson, E., Helsing, D., Kegan, R., Popp, N., Broderick, M., & Portnow, K. (2001). The

Power of a Cohort and of Collaborative Groups. Focus on Basics: Connecting Research

and Practice, 5(B), 15-22.

Dyson, L., & Hanley, B. (2002). Testing the Effect of a Cohort Grouping Model as a Form of

Instructional Grouping Teacher Education. Canadian Journal of Higher Education,

32(2), 27-46.

Eder, D., & Enke, J. L. (1991). The Structure of Gossip: Opportunities and Constraints on

Collective Expression Among Adolescents. American Sociological Review, 56(4), 494-

508.

Education Consortium. (2006). Are they really ready to work? Employers' perspectives on the

basic knowledge and applied skills of new entrants to the 21st century U.S. workforce.

U.S.A.: The Conference Board, Inc., Partnership for 21st Century Skills, Corporate

Voices for Working Families and the Society for Human Resource Management.

Eisenberg, M. B., Lowe, C. A., & Spitzer, K. L. (2004). Information literacy: Essential Skills for

the Information Age. Westport, CT: Libraries Unlimited.

Ennis, R. (1985). Critical thinking and the curriculum. National Forum, 65(1), 28-31.

27

Furrer, C., & Skinner, E. (2003). Sense of Relatedness as a Factor in Children's Academic

Engagement and Performance. Journal of Educational Psychology, 95(1), 148-162.

Ghoshal, S. (2005). Bad Management Theories Are Destroying Good Management Practices.

Academy of Management Learning and Education, 4(1), 75-91.

Glaser, E. (1985). Critical thinking: Educating for responsible citizenship in a democracy.

National Forum, 65(1), 24-27.

Gordon, J., Ludlum, J., & Hoey, J. J. (2008, February). Validating NSSE Against Student

Outcomes: Are They Related? Research in Higher Education, 49(1), 19-39.

Grolnick, W., & Ryan, R. (1987). Autonomy in Children's Learning: An Experimental and

Individual Difference Investigation. Journal of Personality and Social Psychology, 52,

890-898.

Harter, S. (1992). The Relationship Between Perceived Competence, Affect, and Motivational

Orientation within the Classroom: Process and Patterns of Change. In A. Boggiano & T.

Pittman (Eds.), Achievement and Motivation: A Social-Developmental Perspective.

Cambridge: Cambridge University Press.

Hausmann, L. R. M., Schofield, J. W., & Woods, R. L. (2007). Sense of Belonging As a

Predictor of Intentions to Persist Among African American and White First-Year College

Students. Research in Higher Education, 48, 803-839.

Hausmann, L. R. M., Ye, F., Schofield, J. W., & Woods, R. L. (2009, 17 April). Sense of

Belonging and Persistence in White and African American First-Year Students. Research

in Higher Education, 50(7), 649-669.

Hayamizu, T. (1997). Between Intrinsic and Extrinsic Motivation. Japanese Psychological

Research, 39, 98-108.

Hurtado, S., Han, J. C., Saenz, V. B., Espinosa, L. L., Cabrera, N. L., & Cerna, O. S. (2007).

Predicting Transition and Adjustment to College: Biomedical and Behavioral Science

Aspirants’ and Minority Students’ First Year of College. Research in Higher Education,

48, 841-886.

Hurtado, S., & Carter, D. F. (1997, October). Effects of College Transition and Perceptions of

the Campus Racial Climate on Latino Students' Sense of Belonging. Sociology of

Education, 70(4), 324-345.

Inkelas, K. K., Daver, Z. E., Vogt, K. E., & Leonard, J. B. (2007, June). Living-Learning

Programs and First-Generation College Students’ Academic and Social Transition to

College. Research in Higher Education, 48(4), 403-434.

Inkelas, K. K., & Weisman, J. L. (2003, May/June). Different by Design: An Examination of

Student Outcomes Among Participants in Three Types of Living-Learning Programs.

Journal of College Student Development, 44(3), 335-368.

Jaffee, D. (2007). Peer Cohorts and the Unintended Consequences of Freshman Learning

Communities. College Teaching, 55(2), 65-71.

James, P. A., Bruch, P. L., & Jehangir, R. R. (2006, Spring). Ideas in Practice: Building Bridges

in a Multicultural Learning Community. Journal of Developmental Education, 29(3), 10-

18.

Kelly, S., & Dietrich, A. (1995, November). The Influence of Program Structure and Learner

Characteristics on Teacher Training Outcomes. Presented at the Mid-South Educational

Research Association, Biloxi, MS.

Kerlinger, F. N. (1986). Foundations of Behavioral Research, 3d Ed.. New York: Holt, Rinehart

and Winston.

28

King, P. M., & Kitchener, K. S. (1994). Developing reflective judgment: Understanding and

promoting intellectual growth and critical thinking in adolescents and adults. San

Francisco, CA.: Jossey-Bass.

King, P. M., & Kitchener, K. S. (2004). Reflective Judgment: Theory and Research on the

Development of Epistemic Assumptions Through Adulthood. Educational Psychologist,

39(1), 5-18.

Klein, S. P., Kuh, G. D., Chun, M., Hamilton, L., & Shavelson, R. (2005). An Approach to

Measuring Cognitive Outcomes Across Higher Education Institutions. Research in

Higher Education, 46(3), 251-276.

Kowal, J., & Fortier, M. S. (1999). Motivational Determinants of Flow: Contributions from

Self-Determination Theory. The Journal of Social Psychology, 139(3), 355-368.

Kuh, G. D. (2001). Assessing what really matters to student learning outcomes: Inside the

National Survey of Student Engagement. Change, 33(3), 10-17, 66.

Kuh, G. D. (2007). Director's Message: If We Could Do One Thing.. In National Survey of

Student Engagement Annual Report. Bloomington, IN: Indiana University Center for

Postsecondary Research.

Kutner, M., Greenberg, E., Jin, Y., Boyle, B., Hsu, Y.-c., Dunleavy, E. et al. (2007). Literacy in

Everyday Life: Results from the 2003 Assessment of Adult Literacy. U.S. Department of

Education: Washington, DC, April 2007. Retrieved 17 November 2007, from Institute of

Education Sciences, National Center for Education Statistics:

http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2007480.

Lipman, M. (1988). Critical thinking: What can it be? Educational Leadership, 45(1), 38-43.

MacGregor, J. (1991). What Differences Do Learning Communities Make? Washington Center

News, 6, 4-9.

Maher, M. (2005). The Evolving Meaning and Influence of Cohort Membership. Innovative

Higher Education, 30(3), 195-211.

Mather, D., & Hanley, B. (1999). Cohort Groupings and Preservice Teacher Education: Effects

on Pedagogical Development. Canadian Journal of Education, 24(3), 235-250.

McFarland, D. A. (2001). Student Resistance: How the Formal and Informal Organization of

Classrooms Facilitate Everyday Forms of Student Defiance. American Journal of

Sociology, 107(3), 612-78.

Meyers, L. S., Gamst, G. C., & Guarino, A. J. (2006). Applied Multivariate Research: Design

and Interpretation. Thousand Oaks, CA: Sage Publications.

Milner, M. (2004). Freaks, Geeks and Cool Kids: American Teenagers, Schools and the Culture

of Consumption. New York: Routledge.

Miserandino, M. (1996). Children Who Do Well in School: Individual Differences in Perceived

Competence and Autonomy in Above-Average Children. Journal of Educational

Psychology, 88, 203-214.

NLCLEAP. (2007). College Learning for the New Global Century. The Association of American

Colleges and Universities, The National Leadership Council for Liberal Education &

America's Promise. Retrieved 17 November 2007, from The Association of American

Colleges and Universities:

http://www.aacu.org/advocacy/leap/documents/GlobalCentury_final.pdf.

Norris, C. J., & Barnett, B. (1994, 28-30 October). Cultivating a New Leadership Paradigm:

From Cohorts to Communities. Annual Meeting of the University Council for

Educational Administration, Philadelphia, PA.

29

NSSE. (2005). Exploring Different Dimensions of Student Engagement: 2005 Annual Survey

Results (Kuh, G. D., Ed.). Retrieved 20 October 2007, from

http://nsse.iub.edu/pdf/NSSE2005_annual_report.pdf.

NSSE. (2007). 2002 Psychometric Framework. In National Survey of Student Engagement.

Retrieved 1 May 2008, from Indiana University Center for Postsecondary Research:

http://nsse.iub.edu/html/psychometric_framework_2002.cfm.

Pace, C. (1984). Measuring the Quality of Student Experiences. Los Angeles, CA: University of

California, Higher Education Research Institute.

Pascarella, E. T. (2001, Sep/Oct). Using Student Self-Reported Gains to Estimate College

Impact: A Cautionary Tale. Journal of College Student Development, 42(5), 488-492.

Pascarella, E. T., & Terenzini, P. (1991). How College Affects Students: Findings and Insights

from Twenty Years of Research. San Francisco: Jossey-Bass.

Pastors, C. R. (2006, May/June). Assessing a Learning Community Program Through a Student

Survey. Assessment Update, 18(3), 1-8.

Perry, W. G. (1970). Forms of Intellectual and Ethical Development in the College Years: A

Scheme. New York: Holt, Rinehart & Winston.

Pike, G. R. (2000). Assessment measures: Methodological issues in the assessment of learning

communities. Assessment Update, 12(2), 14-15.

Pike, G. R. (2006a, September/October). The Convergent and Discriminant Validity of NSSE

Scalelet Scores. Journal of College Student Development, 47(5), 551-564.

Pike, G. R. (2006b). The Dependability of NSSE Scalelets for College- and Departmental-level

Assessment. Research in Higher Education, 47, 177-195.

Pike, G. R., Schroeder, C., & Berry, T. (1995). The Relationships Between Self Reports of

College Experiences and Achievement Test Scores. Research in Higher Education,

36(1), 1-22.

Radencich, M., Thompson, T., Anderson, N., Orapallo, K., Fleege, P., Harrison, M. et al. (1998).

The Culture of Cohorts: Pre-Service Teacher Education Teams at a Southeastern

University in the United States. Journal of Education for Teaching, 24(2), 109-28.

Reis, H. (1994). Domains of Experience: Investigating Relationship Processes from Three

Perspectives. In R. Erber & R. Gilmour (Eds.), Theoretical Frameworks for Personal

Relationships (pp. 87-110). Hillsdale, NJ: Erlbaum.

Reynolds, K. C., & Sitharaman, S. (2000). Business education in cohorts: Does familiarity breed

learning? Journal of Business and Training Education, 9, 29-ff.

Ridgeway, C. (1983). The Dynamics of Small Groups. New York: St. Martin's.

Roksa, J. (2010, February). Bachelor’s Degree Completion Across State Contexts: Does the

Distribution of Enrollments Make a Difference? Research in Higher Education, 51(1), 1-

20.

Rosenthal, R., & Rosnow, R. (1991). Essentials of behavioral research: Methods and data

analysis. New York, NY: McGraw-Hill.

Ross, D., D., Stafford, L., Church-Pupke, P., & Bondy, E. (2006). Practicing collaboration:

What we learn from a cohort that functions well. Teacher Education and Special

Education, 29(1), 32-43.

Ryan, R. M., & Deci, E. L. (2002). Overview of Self-Determination Theory: An Organismic

Dialectical Perspective. In E. L. Deci & R. M. Ryan (Eds.), Handbook of Self-

Determination Research (pp. 3-33). Rochester, N.Y.: University of Rochester Press.

30

Ryan, R. M., & Grolnick, W. (1986). Origins and Pawns in the Classroom: Self-Report and

Projective Assessments of Individual Differences in Children's Perceptions. Journal of

Personality and Social Psychology, 52, 550-558.

Ryan, R. M., Stiller, J., & Lynch, J. (1994). Representations of Relationships to Teachers,

Parents, and Friends as Predictors of Academic Motivation and Self-Esteem. Journal of

Early Adolescence, 14, 226-249.

Ryan, R. M., Connell, J., & Deci, E. L. (1985). A Motivational Analysis of Self-Determination

and Self-Regulation. In C. Ames & R. Ames (Eds.), Research on Motivation in

Education (pp. 13-51). New York: Academic Press.

Ryan, R. M., & Deci, E. L. (2000). Self-Determination Theory and the Facilitation of Intrinsic

Motivation, Social Development, and Well-Being. American Psychologist, 55(1), 68-78.

Saltiel, I. M. (1994). Support Systems: A Comparison of Factors That Influence Adult Doctoral

and Undergraduate Students Who Are Employed Full Time.

Saltiel, I. M., & Russo, C. S. (2001). Cohort programming and learning: Improving educational

experiences for adult learners. Malabar, FL: Krieger.

Sapon-Shevin, M., & Chandler-Olcott, K. (2001, November/December). Student Cohorts:

Communities of Critique or Dysfunctional Families? Journal of Teacher Education,

52(5), 350-64.

Schmuck, P. (1988). Preparing Superintendents for the Unexpected, the Unanticipated and the

Untoward. ERIC Document Reproduction Service No. ED 309 548.

Seifert, K., & Mandzuk, D. (2006). Student cohorts in teacher education: Support groups or

intellectual communities. Teacher College Record, 108(7), 1296-1320.

Shapiro, N. S., & Levine, J. H. (1999). Creating Learning Communities. San Francisco: Jossey-

Bass.

Standage, M., Duda, J. L., & Ntoumanis, N. (2003). A Model of Contextual Motivation in

Physical Education: Using Constructs from Self-Determination and Achievement Goal

Theories to Predict Physical Activity Intentions. Journal of Educational Psychology, 95,

97-110.

Stassen, M. L. A. (2003, October). Student Outcomes: The Impact of Varying Living-Learning

Community Models. Research in Higher Education, 44(5), 581-613.

Stipek, D. J. (1988). Motivation to Learn: From Theory to Practice. Englewood Cliffs, NJ:

Prentice Hall.

Taylor, P. (2003, 23 June). Epic: Blended Knowledge and Learning Solutions. Retrieved 9

February 2007, from Epic Performance Improvement Limited:

http://www.epic.co.uk/content/news/archive/2003/news_stories_2003/ns_230603_01.htm

l.

Tinto, V. (1987). Leaving College: Rethinking the Causes and Cures of Student Attrition.

Chicago: University of Chicago Press.

Tinto, V. (1988). Stages of student departure: Reflections on the longitudinal character of student

learning. Journal of Higher Education, 59(4), 438-55.

Tinto, V. (1997). Classrooms as Communities: Exploring the Educational Character of Student

Persistence. Journal of Higher Education, 68(6), 601-23.

Tinto, V., Goodsell, A., & Russo, P. (1993). Building Community Among New College

Students. Liberal Education, 79(4), 16-21.

31

Tinto, V., Goodsell, A., & Russo, P. (1994). Building Learning Communities for New College

Students: A publication of the National Center on Postsecondary Teaching, Learning and

Assessment, Pennsylvania State University.

Vallerand, R., & Bissonnette, R. (1992). Intrinsic, Extrinsic and Amotivational Styles as

Predictors of Behavior: A Prospective Study. Journal of Personality, 60, 599-620.

Vansteenkiste, M., Lense, W., & Deci, E. L. (2006). Intrinsic Versus Extrinsic Goal Contents in

Self-Determination Theory: Another Look at the Quality of Academic Motivation.

Educational Psychologist, 41(1), 19-31.

Wesson, L., Holman, S., Holman, D., & Cox, D. (1996). Cohesion or Collusion: Impact of a

Cohort Structure on Educational Leadership Doctoral Students. Presented at the

American Educational Research Association, New York.

Wheelan, S. A., & Lisk, A. R. (2000). Cohort group effectiveness and the educational

achievement of adult undergraduate students. Small Group Research, 31(6), 724-738.

White, R. (1959). Motivation Reconsidered: The Concept of Competence. Psychological Review,

66, 297-333.

Wolcott, S. K. (2006). College Faculty Handbook: Steps for Better Thinking. Unpublished.

First draft available swolcott@wolcottlynch.com.

Zhao, C.-M., & Kuh, G. D. (2004). Adding Value: Learning Communities and Student

Engagement. Research in Higher Education.

32

Figure 1. An SDT-Informed Model of Cohort Effects

Mediating Variable

Response

Variables Explanatory Variable

Control Variables

Academic

Development

Job

Preparation

Enrichment Activities

HOT Assignments

SAT Scores

Demographics (Gender, Age, Ethnicity)

Educational

Outcomes

Learning Community Participation

Relatedness

33

Figure 2. Mediating Effects of Relatedness Where X Represents Cohort Participation,

Z Represents Relatedness, and Y Represents Each DV

X Z Ya b

c

34

Table 1 Description and operationalization of variables

Variable Name and

Construct Definition Operationalization of NSSE 2005 Items

Cronbach’s

Alpha

Endogenous Explanatory

Variable

Cohort / learning community

participation:

Participated in a learning community or some

other formal program in which groups of

students took two or more classes together.

(Coded as a binary variable).

(categorical)

Endogenous Explanatory

Variable (Mediating)

Relatedness: Feelings of

belongingness and

connectedness with others.

During the current school year, the extent to

which students used e-mail to communicate

with an instructor.

During the current school year, the extent to

which students discussed grades or

assignments with an instructor.

During the current school year, the extent to

which they discussed ideas from their readings

or classes with faculty members outside of

class.

During the current school year, the extent to

which they worked with faculty members on

activities other than coursework (committees,

orientation, student life activities, etc.)

The quality of their relationships with other

students at the institution.

The quality of their relationships with faculty

members at the institution.

.688

Response Variables

Academic development:

Ability to read, write, think

critically, and learn

independently.

Extent to which the institution contributed to

student’s writing clearly and effectively.

Extent to which the institution contributed to

student’s speaking clearly and effectively.

Extent to which the institution contributed to

student’s thinking critically and analytically.

Extent to which the institution contributed to

student’s learning effectively on his/her own.

.825

Job preparation: Extent to

which student is prepared to

enter professional

employment.

Extent to which the institution contributed to

student’s job- or work-related knowledge and

skills.

Extent to which the institution contributed to

student’s working effectively with others.

Extent to which the institution contributed to

student’s solving of complex real-world

problems.

.727

Exogenous Explanatory

35

Variables (Control)

Higher order thinking (HOT)

assignments During the current school year, the extent to

which coursework emphasized working on a

paper or project that required integrating ideas

or information from various sources.

During the current school year, the extent to

which coursework emphasized putting together

ideas or concepts from different courses when

completing assignments or during class

discussions.

During the current school year, the extent to

which coursework emphasized analyzing

elements of an idea, experience, or theory.

During the current school year, the extent to

which coursework emphasized synthesizing

and organizing ideas, information, or

experiences into new, more complex

interpretations and relationships.

During the current school year, the extent to

which coursework emphasized making

judgments about the value of information,

arguments, or methods.

During the current school year, the extent to

which they examined the strengths and

weaknesses of their own views on a topic or

issue.

.762

Enrichment activities Participated in a practicum, internship, field

experience, co-op experience, or clinical

assignment.

Participated in community service or volunteer

work.

Worked on a research project with a faculty

member outside of course or program

requirements.

n/a

SAT scores Incoming college entrance exam scores as reported

by institutions, adjusting ACT scores to SAT

equivalents as required.

n/a

Gender (males coded as 1;

females as 2)

Self-reported gender identification. (categorical)

Age (dichotomized: 20-23

coded as 0; 24-29 coded as 1)

Age in years as reported by participating

institution.

(categorical)

Ethnicity (dichotomized:

white coded as 0; non-white

as 1)

Self-reported racial or ethnic identification. (categorical)

36

Table 2 Frequencies and percentages for categorical variables

Cohort Participation

No Yes

Student Characteristic N % N %

Gender

Male 458 49.80 462 49.50

Female 461 50.20 471 50.50

Ethnicity

White 775 84.30 763 81.80

Other 144 15.70 170 18.20

Age

20-23 802 87.30 824 88.30

24-29 117 12.70 109 11.70

Total 1,852 usable records

Table 3 Mean differences between cohort participants and nonparticipants on selected measures

Cohort Participation

No Yes

Variable N Mean N Mean

Mean

Difference

Std dev

(Total)

Effect

Size

Higher order thinking

assignments (HOT)

918

17.61 932 18.91 1.29*** 3.25 0.40

Academic development 918 12.14 932 12.84 0.71*** 2.67 0.27

Job preparation 919 8.93 931 9.58 0.64*** 2.07 0.31

Relatedness to faculty

and students

917 20.96 933 22.80 1.84*** 3.79 0.49

Enrichment activities 919 0.59 932 0.98 0.38*** 0.79 0.49

SAT scores 919 1067.58 933 1047.85 -19.73** 160.81 0.12

***p ≤ .001, **p ≤ .01.

Table 4 Simple effect of cohort participation on educational outcomes N=1,850

Outcome Variable B β

Std dev

Y-Standardized

Effect Size R2

Academic Development .707 .133*** 2.67 0.26 0.018

Job Preparation .643 .155*** 2.07 0.31 0.024

***p ≤ .001.

37

Table 5 Results of block-entry regression of relatedness on academic development N=1,846

Block 1 Block 2 Block 3

Independent variable B β B β B β

Constant 7.316 7.290 4.855

Gender 0.338 .063** 0.341 0.064** 0.353 0.066**

SAT -0.002 -.120*** -0.002 -0.119*** -0.002 -0.105***

Age -0.478 -.059** -0.473 -0.058** -0.36 -0.044*

Ethnicity 0.266 0.037 0.262 0.037 0.347 0.049*

Enrichment activities 0.159 .047* 0.142 0.042 0.015 0.005

Higher order thinking 0.365 .445*** 0.362 0.442*** 0.256 0.313***

Cohort participation 0.123 0.023 -0.047 -0.009

Relatedness 0.195 0.277***

Model 1: R2 = .230, Block 2: R

2 = .231, ΔR

2 = .001; Block 3: R

2 = .284, ΔR

2 = .053.

***p ≤ .001, **p ≤ .01, *p ≤ .05.

Table 6 Results of block-entry regression of relatedness on job preparation N=1,845

Block 1 Block 2 Block 3

Independent variable B β B β B β

Constant 5.963 5.914 3.403

Gender 0.150 0.036 0.156 0.038 0.168 0.040*

SAT -0.001 -.109*** -0.001 -0.106*** -0.001 -0.088***

Age -0.591 -.093*** -0.581 -0.092*** -0.466 -0.073***

Ethnicity -0.033 -0.006 -0.039 -0.007 0.051 0.009

Enrichment activities 0.223 .085*** 0.192 0.073*** 0.060 0.023

Higher order thinking 0.244 .383*** 0.239 0.374*** 0.130 0.204***

Cohort participation 0.230 0.055* 0.056 0.014

Relatedness 0.201 0.367***

Block 1: R2 = .185, Block 2: R

2 = .188, ΔR

2 = .003; Block 3: R

2 = .281, ΔR

2 = .094.

***p ≤ .001, *p ≤ .05.

38

Table 7 Results of analysis of relatedness as mediator of learning community effects on

educational outcomes

Academic Development Job Preparation

Cohort Coefficient without Relatedness 0.123 0.230

Cohort Coefficient with Relatedness -0.047 0.056

Difference 0.17 0.174

Standard Errorindirect 0.033 0.032

tindirect 5.05 5.3

Significance p < .001 p < .001

Table 8 Overview of effectiveness of cohort and relatedness

Explanatory Variables Response Variables

Academic

Development

Job

Preparation

Cohort (simple effect) Yes Yes

Cohort with control variables No Yes

Cohort with control & mediating (relatedness) variables No No