"Embedded Traditions, Uneven Reform: The Place of the Comprehensive Exam in Composition and Rhetoric...

Heidi Estrem Brad E. Lucas

Eastern Michigan University Texas Christian University

Embedded Traditions, Uneven Reform: The Place of the Comprehensive Exam in

Composition and Rhetoric PhD Programs

Sound doctoral pedagogy, in addition to other forms of professionalization in

PhD work, is essential in nurturing future generations of scholars in composi tion and rhetoric. Using the comprehensive exam as a focal point, this article

identifies absences and contradictions in the field's approach to evaluating the

competency of doctoral students.

I. Traditions and National Perspectives

It is a commonplace among scholars that obtaining the PhD requires surviv

ing certain rites of passage, one of which is the comprehensive exam.1 The say

ing goes something like this: You will never feel as current with scholarship in

your discipline as you did on the day of your exams. Whether or not this is true, at least that's the adage. Passing the exams does not guarantee a doctoral student

the PhD, but it marks one form of approval that "proves" graduate students are

worthy of the discipline. Within most composition and rhetoric PhD programs, assessment of student work varies: seminar papers, qualifying and/or compre

hensive exams, dissertations, and the dissertation defense. While some scholars

in composition and rhetoric have studied the general practices and procedures of

the dissertation, few details about the comprehensive exam have entered disci

plinary discussions about graduate pedagogy, institutional reform, or writing as

sessment. We hope to initiate discussions of the exam as a means of reconsider

ing the evaluation of doctoral students in composition and rhetoric graduate

programs.

Over the past 30 years, many PhD programs in composition and rhetoric

have worked within confines of local traditions tied to older literature PhD pro

grams. Some have found room for rethinking PhD requirements. With the pres sure to professionalize, many graduate students now participate in administrative

work, and written work is often directed to a larger audience than faculty readers

396 Rhetoric Review, Vol. 22, No. 4, 2003, 396^116

Copyright ? 2003, Lawrence Erlbaum Associates, Inc.

This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/page/info/about/policies/terms.jsp

Embedded Traditions, Uneven Reform 397

at the department level. In many programs student work is no longer mere exer

cise or exploration: Seminar papers are configured as conference presentations

or journal articles, and the dissertation is framed as a draft of a future book. De

spite these adjustments in PhD requirements, exams for graduate students in

composition and rhetoric seem to have stayed surprisingly stable over the past 30 years. Through examining specifically the practices surrounding the compre hensive exam, we hope, as Marcy Taylor and Jennifer Holberg do, to "(dislo cate the easy narrative of professionalization" so prevalent in composition (608). The comprehensive exam is a concrete feature of doctoral programs, one that is

both intensely local and generically national at the same time?one that allows

us to pinpoint the intersections between theory and practice. In this essay we first lay out a brief sense of the historical role of compre

hensive exams within PhD programs in the United States. This history provides a contextual frame for our survey of comprehensive exam practices within com

position and rhetoric PhD programs in 2000. We highlight the programs imple

menting positive changes and discuss the tremendous difficulties inherent in

changing a practice so steeped in tradition, hierarchy, and academic rigor.

Throughout this article we do not intend to place blame on anyone, for we un

derstand the thoughtful and time-consuming work that went into establishing PhD programs in composition and rhetoric?and that continues to sustain them now. But as we continue to review discussions of the comprehensive exam, we

wonder why composition and rhetoric scholars?so well versed in writing as

sessment models and historically resistant to traditional views of students and

writing?have not done more to challenge the traditions of work within PhD

programs.

In Refiguring the Ph.D. in English Studies, Stephen North explains a

"magisterial curriculum" as the process by which potential scholars are selected

and tested to become members of an academic community. The magisterial cur

riculum?one that continues to flourish?reaches its summit with "barrier" ex

aminations that North describes as an "obstacle course" and "ritual gauntlet" for

graduate students (30-33). According to North, such examinations generally share the same characteristics:

They were offered under a variety of names?qualifying, prelimi nary, comprehensive?and assumed a variety of forms, as well. They

might, for example, have been exclusively oral, exclusively written, or (most often) some combination of the two; might have been given over several hours, or carried on for two or more days; might have been administered by a particular committee, or been open to all



398 Rhetoric Review

interested faculty; they might have given the candidates some choice over the "areas" to be examined, but might not, and so on. (33)

He contends that these examinations, though rigorous, did not (and do not, he

suggests) comprise the full rite of passage; instead, he asserts that the examina

tions must be validated by a faculty sponsor who will testify to the candidate's

acceptability (35). In other words, intelligence is necessary, but without the sup

port of a faculty member, a graduate student cannot be granted candidacy.

North's conclusion points to the murky relations between graduate students and

their faculty (a topic beyond the scope of this discussion), but the vagueness sur

rounding the exams prompts questions about the nature of the exam itself. How

did the comprehensive exam emerge, what is it today, and what informs its cur

rent use as an evaluative practice?

II. Increasing Evaluations and Assessments:

"Cooling Out" Structures

"All around us," writes Robert Connors, "are the data of the present, and

they constantly press on us the immediate question, 'What shall we do?'" To

gain perspective we briefly take up the "historian's questions" that Connors

cites: "'What have people done in the past?' and 'How did things come to be this

way?"' ("Dreams" 17). In the late nineteenth century, the American dissertation was established simply as a marker of difference between the BA and PhD de

grees.2 The Association of American Universities (AAU) in 1900 began to work

toward uniformity in doctorate requirements, occasionally addressing admis

sions standards (Geiger, Advance 18). In the 1920s, the number of PhDs nearly

tripled, increasing by another 50 percent in the 1930s, but there were still fewer

graduate students than most programs wanted (Geiger, "Research" 245^-6). Na

tionwide efforts at standardization were few, perhaps because of the inherent

challenges. For example, one Carnegie Foundation researcher found graduate education frustrating, confusing, and overwhelmingly difficult to describe at the

national level; he concluded that individual departments held sole responsibility for standards (249).

With rising enrollments and new programs established at other universities, schools like Harvard began restricting graduate admissions in 1930. Schools

would later curtail admissions with the help of the Graduate Record Examina

tion (GRE) established in 1937 (Geiger, "Research" 249). Whereas the disserta

tion was initially a mechanism for managing student populations, the growth of

doctoral education prompted more and various forms of evaluation. By the late

1930s, as Roger Geiger explains, "the potential university teacher was subject to

evaluative hurdles at recurrent intervals" (250). Shortly thereafter, the structure




of PhD education became more or less consistent: "[A] student passed a compre hensive examination, acquired a reading knowledge of French and German,

handed in a satisfactory dissertation, and survived a final oral examination"

(Stanford 244). While the overall progression remains more or less intact today, the emphasis on evaluation began to shift after World War II.

The GI Bill and American competition with Soviet science prompted a sec

ond wave of university growth in the 1960s, tripling the number of PhDs earned

and leading to an increasingly impersonalized graduate school experience (Gei

ger, "Research" 253). Some of the first critiques of the exam emerged in the late

1960s but were mostly in response to administrative and programmatic ineffi

ciency?not from considering students (Beck and Becker). To streamline the in

stitutional processes for faculty and students, eventually many of the end-stage evaluations became less onerous, and many programs began to move the barrier

examinations to an earlier stage in a student's graduate career (Mayhew and

Ford 237). This, of course, took much of the pressure off the dissertation exams.

In the early 1970s, Bernard Berelson noted the lack of rigor in the final examina

tion?the dissertation defense?which had decayed into an exercise serving only a "ritualistic function" (230). He contrasted this to the "qualifying, comprehen sive examination" that had been "moved up in time" to "an earlier, less culmi

nating point" from the dissertation defense; this comprehensive exam is "not

only a real examination in substance and importance but one that contributes,

through his preparation for it, to the candidate's breadth within his own disci

pline" (199, 217). By the mid-1970s, the total number of American PhDs

awarded annually hit a plateau?approximately 34,000?that would remain somewhat consistent until the late 1980s (Geiger, "Research" 255). Calls for

abolishing the exam (Jako) were tempered by other calls to reform and redefine

the exam to reflect more realistic aims (Wolensky), but ultimately few changes were effected.

With shifts in evaluation emphases, doctoral programs not only assumed

gatekeeping mechanisms but also created attrition rates scandalously higher than

those for undergraduate programs. Robert Cowen calls the American PhD "a

highly structured experience" with "a considerable sequence of controls over the

timing and pacing of the preparation?over access to the start line"; however, "each of the quality control points means the possibility of an individual being 'cooled out' of the system in the first couple of years" (193). In one recent na

tional survey, roughly 50 percent of student respondents agreed that "evaluation

of students successfully 'weeds out' weak doctoral students" (Anderson and

Swazey 5). The process of "cooling out" or "weeding" students from programs is not the sole reason for attrition but certainly contributes to the high rates. Con

trary to the popular belief that the dissertation poses the greatest hurdle, one ma

jor study determined that "the majority of the graduate students who failed to



400 Rhetoric Review

earn their doctorates left the program before advancement to candidacy, not

after" (Nerad and Cerny 340). In their massive 1992 study, William Bowen and Neil Rudenstine conclude

that this middle-stage between coursework and the dissertation

has become unusually difficult. [. . . A] less well-defined interim pe riod [... that] may involve passing a general examination or qualify

ing exam and satisfying foreign language requirements, as well as

choosing a dissertation topic and, in some cases, submitting an ap

proved prospectus. (251, 254)

In fact, as one UCLA study revealed, roughly 30 percent of students leave dur

ing the early period of their doctoral studies (Nerad and Cerny 340). Other stud

ies concur, noting that roughly 25 percent of students at larger programs (10-15

percent at smaller ones) do not return for a second year, and out of those who do

return, roughly 30 percent do not reach ABD status (Bowen and Rudenstine

252-53). Of the doctoral students who do pass their exams, approximately 70

percent do not complete the degree (Ziolkowski 185). Various factors influence

the decision to leave, with 40-50 percent noting "burn-out" as the most promi nent factor in not completing their degrees (256), but one thing is clear: The

comprehensive exam not only comes at a crucial point in a student's graduate

career but is also imbued with a sense of institutional and programmatic selec

tion conjoined with high rates of attrition.

What has been noted?usually only in passing?is the lack of consistency and clarity about this middle-stage work that allows a student to cross over into

ABD status. Some researchers have noted students' lack of understanding about

the examination itself. Maresi Nerad and Joseph Cerny note that one problem is

student overpreparation for the exam:

Their ideas about the structure, scope, and standards of the exam

were vague. The aura of mystery which surrounded the exam

seemed to lead to a self-imposed perfectionism. They were disap

pointed, after taking the exam, to discover that they were given the

opportunity to demonstrate only a small part of their knowledge. [. . .] After passing their orals, they had trouble mustering enough

energy to write a dissertation proposal. (342)

If "burn-out" is the major factor contributing to attrition, then the exam may

play a larger role than has been considered in the past. Replacing the exam

with other mechanisms, however, does not necessarily ensure students will




pass through the middle stage with less trepidation. For example, one political science program took the route of offering a formal writing option, instead of

examinations: Students submitted two papers instead of taking a preliminary examination, and as a result, the papers became "mini dissertations" in stu

dents' minds, resulting in protracted, indefinite revising and, as Bowen and

Rudenstine reveal, "it took several years, with almost total attrition, before the

situation was addressed" (266). Within composition and rhetoric scholarship, there have been a few calls to

rethink the PhD process. In Composition-Rhetoric Connors points to the exams

as a "vestigial practice" and "a last agonistic remnant" of a male-dominated aca

demic culture that perceives life as combat, an ongoing contest over power, terri

tory, and prestige: "[T]he 'defense' of the PhD dissertation or master's thesis is a

curious relic, an atrophied survival of a harder time no one remembers" (25, 47). However outdated such practices might seem, they do persist in newer forms, based now more heavily on the relationship a student fosters with her or his ad

visory committee?raising concerns that, for some students, the comprehensive

exam is more of a social contest than a measure of academic competency.

III. Assessment Traditions and Enacted Theory

Over the past 30 years, composition and rhetoric PhD programs have grown in size and number, with new programs currently in development. We wanted to

discover what vestiges of tradition were readily apparent in material descriptions of these programs, and why. To begin to draw the barest map of what underlying theories shaped comprehensive exams in composition and rhetoric programs, we

surveyed PhD program requirements from the 65 composition and rhetoric pro

grams listed in the Spring 2000 issue of Rhetoric Review. Our starting research

question was, simply: What do program documents reflect about the status of

the comprehensive exam in composition and rhetoric PhD programs? First, we

compiled exam information from each program that we could find available on

line, and we supplemented that information with print materials we had gathered for graduate advisement (information on six programs was not available). We

compiled this information into a searchable database. Keeping in mind Keith

Grant-Davie's work on the interpretive nature of coding data, we combed

through the collected materials for a cumulative sense of what the process was

like in all programs ("Coding Data"). After developing a list of key terms and

ideas, we sorted our data so that we could look for national patterns concerning the comprehensive exam.

We realize that likely not all programmatic information about the exam is

publicly available online or in other public documents, but we targeted the



402 Rhetoric Review

accessible documents to question the requirements and reflective practices in

programs across the country. Ideally, we would have liked an extensive survey and interview procedure of faculty and students from each program, and this

would be an excellent "next step" for scholarship in this area. For our purposes the program statements raise intriguing and unsettling questions about what has

changed?and what has not?in the ways composition and rhetoric programs

imagine the exam process today.

We understand the dangers of relying only on program documents for the

whole story, or even part of it. In an issue of Composition Studies focusing on

doctoral pedagogy, Louise Wetherbee Phelps notes:

Every doctoral program in rhetoric and composition expresses and

acts out vividly in concrete features a theory of the discipline: in its

curricular content, requirements, qualifying exams and reading lists,

range or concentration of faculty specializations, acceptable range of

dissertation topics or inquiry methods, brochures, program, and fac

ulty titles. Less explicitly (though this should be transparent to any

compositionist), its pedagogy?teaching methods, curriculum, use

of technologies, mentoring practices, assessment?embodies a phi

losophy of education, including a conception of adult learning and

adult development. (118)

Local discussions are not captured by the "concrete features" of our discipline

(the program descriptions in this case), nor are local tensions stemming from in

dividual professors or graduate students. However, for a field composed of

scholars trained in rhetoric, there are few excuses for ignoring the written em

bodiments of programmatic acts.

In most graduate programs, disciplinary experts conduct the comprehensive exam as a conventional text-performance. The exams, however, extend the tradi

tional assessment roles Kathleen Blake Yancey assigns to testing specialists and

teachers. It is useful to consider the comprehensive exam-as-assessment tool

parallel to first-year writing assessment tools that have been discussed, debated, and refined in our national journals for two decades (Yancey). Graduate educa

tors are guides, gatekeepers, dissertation advisers, seminar leaders?and often

friends and mentors ever willing to discuss, theorize, and reflect as colleagues.

Perhaps not surprisingly, given the traditional lore surrounding the comprehen sive exam, the vast majority of composition and rhetoric PhD programs have ei

ther a written or an oral comprehensive exam, and over half require both (see Ta

ble 1). Since the majority of these exams set out to test expert knowledge, it also

should be no surprise that nearly two-thirds of the programs designate faculty as

the assessors. When they do not indicate who will be assessing the exams, we




Table 1

Administration of Comprehensive Exams

Number of

Programs (N = 59) Percent

Some kind of comprehensive exam 58 98.30

Both written and oral exams 34 57.62

Administered and Evaluated By Not specified 23 38.98

Faculty (committee, departmental, or student-chosen) 36 61.01

wonder if the lack of specification merely reflects a belief that to do so would be

stating the obvious.

Even though the field of composition now acknowledges the usefulness of

portfolios and other means of more authentic assessment of student writing, these established theories about undergraduates seem to be interpreted differ

ently for the graduate setting. After decades of concentrated scholarly effort, a

set of assumptions undergirds most forms of composition instruction and disci

plinary discussions of assessment. First-year writers often work together to draft

and redraft their essays, refining arguments and building community, and sub

mitting portfolios and writing for various audiences. In terms of evaluation, spe

cifically, scholars have developed a clearer sense of how authentic assessment

practices might work: At a minimum, purposes for the assessment should be

clearly articulated, and the assessment criteria should be provided. How, then, are these practices echoed in composition and rhetoric PhD programs?

Within individual classes and seminars, many of these pedagogical practices

likely flourish. But at the crossover point of the doctoral examination, it appears that not much has changed. Despite all we know about assessing student work, out of the 59 programs with public documentation available, only six (10 per cent) require papers, essays, or other compilations of written work (in portfolios or other configurations) to measure their students' knowledge. Only 24 (41 per cent) of these programs indicate that there is an option for students to revise their work. And no programs publicly account for collaborative work. Imagine if our conversations about undergraduate curricula across the country reflected

similar profiles. As David Damrosch explains:

If we really press the question of what would be entailed in fostering collaborative learning at the graduate level, we are likely to be led to

alter every aspect of our programs: course requirements, distribution

requirements, language requirements, and guidelines for the struc

turing of doctoral examination and of the dissertation. (158)



404 Rhetoric Review

Perhaps attrition rates would be lower?and completion rates higher?for doc

toral students if programs would consider what seems to be familiar pedagogy: collaborative work, revision processes, and portfolio evaluation.

Yancey's recent overview of assessment and testing gives us a picture of three

"waves" of assessment in writing.3 The most recent "third wave" of portfolio as

sessment has had the "specific effects of democratizing, localizing, and grounding

expertise of three kinds: student expertise, reader expertise, and theorist expertise"

(496). Even as portfolios unfortunately have the potential to "oppress more than

make possible," they still hold out hope within first-year writing programs in that

they encourage learners to provide themselves commentary and criticism for their

own development as writers (498). Ideally, such third-wave assessment procedures could accomplish similar goals as part of a doctoral student's progression to candi

dacy. However, documents from PhD programs suggest that an understanding of

assessment has swept through the field of first-year writing without leaving many traces in graduate pedagogy. Instead, doctoral exams remain in an assessment form

designed to examine particular disciplinary behavior.

Yancey characterizes this form of "second wave" assessment as dominated by

the holistically scored essay test, which replaced earlier tests of indirect measures

with a direct form of measurement that merely samples the target behavior, other

wise known as student writing (486). A manageable writing sample?produced under various constraints of time, place, and subject matter?becomes a way for

faculty ostensibly to measure student behavior. As Yancey notes, "The question

'How shall we evaluate writing?' concerns itself not only with methodology, but

also with behavior: which behavior should we examine?" (486). In the context of

undergraduate writing pedagogy, we often protest against high-stakes testing that

can define students entering college, overriding their previous record in high school courses. However, we have set up similar expectations for the comprehen

sive exam; as one program's documents read, "[S]tudents should also note that

good coursework grades do not automatically indicate that they are prepared to un

dertake their Comprehensive Exams." The exams, then, test something beyond what students might learn as members of a graduate program; they ostensibly test

behaviors of a soon-to-be professional. If we allow the gains from assessment the

ory to inform doctoral pedagogy, then similar questions can emerge about the prac tices used to evaluate graduate students, and the way they behave: How shall we

evaluate graduate students' writing? What behaviors are we examining, and why? While a majority of programs do not publicize their aims, nearly half (43

percent) of programs do give clear explanations for the purposes behind their

comprehensive exams. Generally, these purposes fall into three categories: criti

cal thinking abilities, expert knowledge, and purposes related to the research or

teaching abilities of graduate students (see Table 2).



Table 2

Stated Purposes of the Comprehensive Exam

Critical Thinking Expert Knowledge Research/Teaching Ability

Critical skills

Ability to make

connections among those areas

Ability to reason critically and creatively

Develop and defend an

informed position on

topics or questions

important to major and

minor areas of study Make a sustained critical

argument about the texts

on the reading list

Ability to criticize works and

relate them to other works

Critical ability An ability to analyze

penetratingly, persuasively, and succinctly

Ability to respond

appropriately Intellectual capacity

Ability to carry on a critical,

scholarly dialogue

Expressing it in coherent

persuasive prose

Integrating and synthesizing

large amounts of

information

The ability to make

substantive connections

among texts

Historical breadth

Appropriate knowledge of

primary and secondary works

A command of the discipline

Knowledge of pertinent

theoretical, methodological, and critical or interpretative

scholarship

Expert knowledge in areas

supported by faculty and

courses

Broad knowledge in three areas

Student's knowledge about

representative texts in

composition and rhetoric

A broad and basic knowledge A level of expertise in rhetoric

to warrant admission to

dissertation work

A mastery of the field that

clearly surpasses that of the

generalist General fundamental knowledge Student's knowledge of primary

texts, contexts, and authors

A working knowledge of

relevant secondary materials

Test overall knowledge Immersion in and conversance

with the academic topics and

literature in the fields of

study

Understanding of the works on

the reading list

Be able to rattle off definitions

of key terms

Mastery of a specialized area of

expertise Tests students' knowledge of

their lists

Student must evince a

specialized knowledge ...

not merely ready use but also

synthesis and potential for

creation

Overall knowledge

Strong familiarity with

important details of the texts

Establishes student's ability to teach and do research

(secondary to knowledge about representative texts)

To determine a candidate's

readiness to begin the

specialized work that leads

to the dissertation

Skills as a researcher,

scholar, and critic

Adequacy of training for

scholarly research

Intended to help you prepare to write your dissertation

by establishing breadth

Augments faculty adviser's

ability to certify to

potential employers the

student's teaching and

research competence



406 Rhetoric Review

Within the constraints of the exam, students need to show their knowledge of an increasingly unknowable field. When the field was much younger, such

feats of comprehension were somewhat manageable. However, even as early as

1984, as Edward Corbett would recall, the first edition of the Bedford Bibliogra

phy for Teachers of Writing prompted new graduate students to feel "at once in

spirited and intimidated. [. . .] New students certainly know that if they decide to

make a professional career of rhetoric and composition, they have a lot of rather

heady reading to do" ("Teaching" 447-48). Eighteen years later, not much has

changed, except that the levels of inspiration and intimidation are perhaps more

intense. As Wendy Bishop remarked during her President's address at the 2001

Conference on College Composition and Communication (CCCC) in Denver, the current generation of scholars cannot keep up with the torrent of scholarship now published in the field. Although the field's breadth defies inclusive knowl

edge, some program documents reflect a desire to have students know every

thing, reflecting a vision of an ideal writer-subject who first knows and then

writes. As one program description reads, "Remember that when you sit down to

write, the last thing you want is to have to think."

From our compilation of program descriptions, only two programs attempt to list clearly the criteria that will be used to assess graduate students. One tech

nological university states that "the quality of the student's work and perfor mance during the exams" will be based on the "quality of the arguments in each

of the four written responses" and the "quality of the thinking and knowledge demonstrated in the oral component of the examinations." Similarly, a women's

university's documents state:

To pass "with distinction," the student's performance must be judged

by a majority of the members of the committee to be exceptional in

quality and to represent a degree of knowledge regarding the areas

of examination that demonstrates, not just mastery of the subject, but

a depth and breadth of sophistication that clearly distinguishes the

performance from that of the majority of examination candidates.

While it is admirable that these two institutions have begun to consider what

they value in assessing students' work on the comprehensive exam, even these

indications give graduate students little sense of how they will know when they have met the assessment criteria. What are markers of quality in an argument? In

thinking and knowledge? In sophistication? Of the composition and rhetoric PhD programs we studied, the remaining

97 percent have no public statement about how exams will be assessed (see Ta

ble 3). Based on our experiences and those shared by others, we can guess that




Table 3

Assessment Criteria for Traditional Comprehensive Exams

Number of

Programs (N -

59) Percent

Assessment Criteria

No criteria listed 57 96.61

Quality of arguments, quality of thinking 1 1.69

Exceptional in quality 1 1.69

Evaluation

No evaluation measures listed 41 69.49

Pass or no pass 5 8.47

Three to five ranks possible 13 22.03

Revision or retake options 24 40.67

Conditional pass with additional coursework or exams 1 1.69

Evaluators

Not specified 23 38.98

Faculty committee (doctoral committee, standing faculty

committee, or unspecified committee) 32 54.23

Student graded anonymously by faculty committee 2 3.38

Graded by anonymous faculty 1 1.69

assessment criteria for graduate students are sometimes delineated more clearly in negotiations with their committees than in program materials. But often, the

criteria do not become clear until after the exam is challenged?or over. And

while assessing graduate students' work by highly nebulous, locally determined

criteria is perhaps the inevitable result of PhD work, it also holds the negative

potential that individually determined criteria for one candidate's work could

differ radically from another's.

A longstanding rite of passage, the comprehensive exam most often posits a

student self as one that has a sense of freedom and agency, but ultimately re

mains bound by a disciplinary hegemony. Imagine, momentarily, that Yancey is

commenting on the comprehensive exam process in this passage:

the self becomes a producer?of a holistically scored essay?and thus an agent who creates text. Still, there is less agency there than it

appears. The text that is created is conventionally and substantively determined?some might say overdetermined?by an expert who

constrains what is possible [. . . T]he authorship of such a text is

likely to be a static, single-voiced self who can only anticipate and

fulfill the expert's expectations. [. ..] At best, agency is limited.

(499)



408 Rhetoric Review

Of these programs that indicated the process of selecting testing content, only 27

(46 percent) allow for student input, and of course, exams are most likely graded by expert faculty (although this is an assumption, since 23 programs [39 percent] do not list who will be assessing the exams, it seems safe to say that faculty do the assessment). In two programs faculty assess anonymous student work: Stu

dents' identities are somehow concealed so that a semblance of objectivity re mains. In one program the faculty are asked to work as anonymous graders (and we wonder just how anonymous they can be). Just as future scholars are about to

embark on their dissertation projects, it seems that in some instances the possi bilities for authentic agency are reduced to compliance with faculty visions of the examined students' answers to faculty-supervised questions.

The majority (70 percent) of programs report no specific evaluation options for the comprehensive exam; it is intriguing to note, however, that despite the dearth of evidence for what the exams will be assessed on, one-fifth of programs (22 percent) do delineate very specifically how students' exams will be ranked and sorted. Some designations for exams include:

No Pass, Conditional Pass, or Pass

Fail, Pass, or Pass with Distinction

Fail, Pass with Reservations, Pass, or Pass with Distinction

Fail, Low Pass, Pass, High Pass, or Honors

Failure with recommendation not to continue in program; failure with per mission to retake exam later; approval to remain in program with conditional

requirements; approval and recommendation to proceed to dissertation Letter grades

According to North, by the time students have reached PhD work in English, they have already been sorted, ranked, evaluated, and classified until only a tiny

minority of students remains (15). The comprehensive exam is then used to fur ther rank and sort the few remaining students completing graduate work, despite attrition rates that have in the past hovered near 50 percent, admissions policies that inevitably bar many interested people from continuing their graduate work, and the self-selection process that students go through before committing to doc toral work. Of course, after earning an undergraduate degree and completing years of graduate-level coursework, most students can rise to this disciplinary challenge and move forward with their careers. But as scholars and teachers, we

must ask ourselves if we are effectively evaluating our future colleagues?or if

we're simply forcing them to comply with our unreflective, unspoken expecta tions based on "what we went through" to get our PhDs.




Almost two decades ago, C. H. Knoblauch and Lil Brannon asked us to con

sider where and how composition classrooms might reflect certain ancient be

liefs about language: "that ideas exist prior to language, that the content of a dis

course is wholly independent of its form, that knowledge is fixed and stable"

(24). They claim that often "the public oration resembled a gymnastic event, where judges know the range of acceptable behavior [.. . ] and evaluate, not

chiefly what is done, but how well a performer does what is supposed to be

done" (25). The preponderance of traditional exams reflected in our survey point to, we're afraid, a type of comprehensive exam that is in line with the "ritual

constraints" of other kinds of traditional school writing, in which each is "de

fined in its pristine generic integrity; the rules for responsible behavior are care

fully depicted; the consequences of disobedience are ominously foretold?vio

late footnote conventions and generations of scholars will groan from their

graves" (29). This set of exams, whether spread over weeks or hours, asks stu

dents to convince readers of both their general and specialized knowledge, all

through a process that does not draw on our knowledge of the dubious results

from such testing situations. These are high expectations of what is most often a

sit-down, timed essay exam (whether 3 hours, 24 hours, or weekend-long). The

performance aspect of these exams is tantamount; students are expected to show

their abilities before an audience of faculty members. Within this unique rhetori

cal situation, one that will never present itself again, doctoral students are sup

posed to show their ability to reason critically and their readiness to do further

critical work?and all expressed in coherent, persuasive prose. The expectations for graduate-level work are reasonable, but the constraints are not. After all, if

graduate faculty insist that quality work can be produced in a few hours, do they not tacitly argue that scholarship is a matter of content-transmission, rather than

careful inquiry, reflection, and socially informed textual processes?

Ultimately, our study of policy documents aims merely to begin assessing the "spirit of the enterprise" of graduate education in composition and rhetoric

PhD programs (North 194). Institutional pressures may be such that the concrete

features we have compiled above have very little relation to the actual work

within programs. The extent to which these program documents belie any at

tempt to accommodate established pedagogical assumptions may point to inertia within these programs, local exigencies, university-wide pressures for "tradi

tional" criteria, or departmental conflicts in cases where composition and litera ture coexist. We list these contingencies and hedges, knowing how true they can

be, and yet we keep returning to this unsettling question: Why, in a young and

progressive field like composition studies, do our public documents reflect a

field that is tied to highly questionable assessment practices? Of course, a new

field also needs to establish itself within the academy, and part of the way this is



410 Rhetoric Review

done is through assuming traditional practices. However, as our PhD programs have grown and matured, perhaps there is now room to begin rethinking what traditions we retain in doctoral education, and why.

IV. Alternative Visions of the PhD Examination

Several graduate programs are seriously reforming graduate pedagogy; the

program descriptions hint at fascinating changes that would be well worth inves

tigating in different venues. As a more detailed, public account of doctoral peda gogy, North's book has been instrumental in helping us place our discussions about graduate pedagogy in another context. While exams at SUNY-Albany are

individually flexible, "governed less by policies than by faculty practices," North

does assert the necessity of policies. They can "establish a context for those ne

gotiations: express, in principles and procedures, the agreed-upon spirit of the

enterprise" (194). In this program the "qualifying" exam is a "writing occasion"

that bridges coursework and the dissertation, serving as "a framework for shap

ing preparation [for the dissertation]" (196). The work and research students do

leading up to the exam results in a three-hour, sit-down writing "occasion" (in the placeholder of an "exam") compared to compulsories in figure skating: "[E] very one involved knows that the performer needs to cover a certain

agreed-upon ground, and do so in a way that allows her or him also to display an

acceptable range of recognizably disciplined movements" (197). Alongside this exam students also write "working papers" over the course of 6-12 months, one

of which outlines a candidate's direction for research.

North's delineation of the processes for the exam and the dissertation point us to an unasked question: Why, in a program where alternative structures are

rich with potential for meaningful alternatives, is there still a sit-down, three-hour comprehensive exam at all? The program certainly holds a lot of pos

sibility for fruitful reconsiderations of how graduate work might be more richly articulated, and we value the honesty of putting a program out for display, espe

cially after we have seen the lack of information about the workings of other

programs.4

Other program documents also hint at changes. At one university, for exam

ple, graduate students submit a Culminating Assessment that consists of either a

writing portfolio or teaching portfolio. Both options include a reflective essay, while the teaching portfolio may also include videos that reflect theory and prac tice in composition. Further, both are presented in a public setting. At another

university the documents provide a hint of intrigue; in addition to "comprehen sive written qualifying examinations," students are "further required to acquire and document a set of competencies and perspectives: historical perspective, use

of technology for scholarship and/or teaching, a research tool or method,




interdisciplinarity in their studies, and professional experience (teaching, intern

ships, appropriate work experience)." We can presume that students will negoti

ate with committee members how to acquire and document these "further" re

quirements, but the program guidelines do not specify how this will happen.

Finally, one university has gone even further by openly abandoning the notion of a "comprehensive" exam altogether, focusing instead on a "PhD Project" that is

"an occasion to integrate various aspects of the changing disciplinary practices of English and to begin to define a broad area of inquiry for subsequent work."

The promise of this project is provocative; yet, interestingly, the exam remains in

the form of two three-hour examinations "based upon the paper and the project,"

culminating in an oral exam.

Before taking full-time positions, we were graduate student colleagues in a

doctoral program that posed an alternative to the comprehensive exam. This

Carnegie I, doctoral-granting institution had offered a composition and rhetoric PhD for almost ten years. Traditionally, its graduate students studying litera ture were given four timed, written exams, followed by an oral examination.

After a few students graduated with emphases in composition and rhetoric, the

program faculty implemented different requirements, and we were part of the first generation to face the new requirements. The newly configured compre hensive "exam" now required an extensive self-directed annotated bibliography and a publication-quality professional paper, culminating in a one-hour "dis cussion" of our material. Like so many graduate students, we went through our

obligatory grumbling and stressful moments, but for us the new process

prompted specific questions about the exam itself. When our discussions needed some grounding, we looked to our program documents for answers,

finding these requirements:

Written: Working closely with the advisory committee, the stu dent will prepare a 30-50 item annotated bibliography of major works in composition and rhetoric. This bibliography is prepared

throughout the student's time in the program. The annotations con

sist of the student's response to readings and analysis of themes and

issues, and they can often lead to ideas for further reading. The can

didate then proposes a topic growing from the reading, and, subject to advisory committee approval, writes a paper of publication qual

ity on that topic. For the additional area of expertise, the student demonstrates proficiency and competence through a written exami

nation or paper, as determined by the committee.

Oral: After the student has completed all aspects of the compre hensive written examinations, the advisory committee will conduct

an oral review of the exam, not to last more than an hour and a half.



412 Rhetoric Review

We understood and believed in some of the ideals behind this exam: that a

cumulative and additive project might encourage reflection during coursework

and beyond, that writing could be used to learn and interrogate that learning, and that a paper for an audience beyond our department would be a useful un

dertaking. And we valued the move toward more authentic means of assess

ment; we knew that timed essays were not fully effective for evaluating student

writers. But we also had our individual doubts about the projects we were de

veloping in lieu of the written comprehensive exam. We found ourselves en

meshed in intense discussions about assessment and evaluation, intrigued to

clarify for ourselves the purposes and aims of evaluation for doctoral candi

dacy. We realized that in a space rich in possibilities for negotiation and

change, we had been part of one program among few that were willing to re

consider the exam process directly.

This alternative structure pointedly critiques traditional exams by relying on

a very different theory of how people learn and write best, but because this alter

native still takes place within the institution, complications remain. As Richard

Miller notes, school work takes place "within the context of a deeply entrenched

bureaucratic system that exercises any number of material constraints on what

must take place in the classroom" and, of course, on what is possible in extracur

ricular work in a program. His supposition about the difference between stu

dents' and teachers' perspectives on liberatory practices unsettles us as teachers

and yet reminds us of how we felt as students: "[W]here teachers see a liberatory

practice and rising opportunities, most students see a set of requirements, an ar

bitrary system of assessment, an impediment to advancement?a bureaucracy, in

short" (19). Doctoral students in this program work with their faculty advisers to

shape their reading lists and form their scholarly essays; ultimately, however, the

faculty sits in judgment, deciding who passes and who doesn't, based on elusive or obfuscated criteria. Tensions of power and control are sidelined or approached

through the language of appropriateness rather than performing to community standards.

Paradoxically, as we worked within this revised structure for the compre hensive exam, the tensions over what was acceptable and what was not became

heightened as the nontraditional manner of these projects became clearer. Ru mors flew among graduate students about individually negotiated annotated bib

liographies of 20 pages while others' projects topped 150. Some faculty seemed

to be unsure of the purpose and parameters of this new project and used its loose

structure to formulate their own requirements for graduate students. We faced

moments of uncertainty about expectations while guidelines for the process were

still in development, but being offered an alternative has led us to think more

critically about both traditional and alternative exam structures. Like Welch we




wonder what happens in the next, postepiphanic scene (and to the players them

selves) when we hear of alternatives implemented successfully. As graduate stu

dents, we often felt the additional burden of performing under a new, sometimes

unwritten set of rules for the comprehensive exam?even while we admired our

faculty advisers' willingness to question the vestiges under which they adminis

tered this evaluative tool.

Behind our narrative here are stories of other graduate students, stories too

fresh to be told since power relationships do not disappear with the granting of a

degree. We remember one student who left the program because of frustrations over the alternative structure. Others chose to be "grandfathered in" under older, traditional exam requirements when they saw other graduate students struggle

with this new program. We've provided these few details with some apprehen sion because they represent individual lives and struggles. And these stories are

not all ours to tell; but perhaps such confessionals are less useful than the issues

they provoke, for it is a challenge to consider the effectiveness of any type of al

ternative exam. It is especially challenging to consider the possibility that such exams can perpetuate the gray areas of the magisterial process. After all, do

alternative exams continue to foster oppression rather than reflection? Do they still encourage static, well-behaved, and singular selves who produce overdetermined, univocal texts? In an attempt to overturn "time-tested" tradition,

do they simply reconfigure tradition, redefine an emphasis on research-oriented

work, and require a new set of rituals, trials, and obstacles?

There are a variety of perspectives about the way we were examined?some

positive, others critical. The alternative exam is still a work in progress, but the

paradox of institutional reproductions abounds, preventing honest assessment:

Just as our faculty mentors struggled to implement these alternative structures within the parameters of an institutional framework, the same professional con

straints provide little room for us to reflect openly on the untold stories here. Too much is at stake among our small community; many of the dilemmas of even

speaking or writing about doctoral pedagogy, then, become once again entangled with issues of power and authority.5

V. Possibilities for Change: Continuing the Conversation

Our own experiences have drawn us to the rich site (as yet underexplored) of doctoral pedagogy. There are promising new directions for inquiry into gradu ate education, but much remains within it to be challenged by graduate students and faculty alike. Perhaps the first step is to recast examination processes into al

ternatives to traditional structures, like those described by North, hinted at in a

few other programs' documents, and undertaken by graduate students in our own



414 Rhetoric Review

PhD program. The next place to turn our collective gaze may be the structures

themselves. If an alternative structure still serves traditional examination and

testing roles, the effects of any newer, more theoretically sound pedagogy may be more detrimental than the processes it attempts to replace.

If we agree with Arthur Applebee that a curriculum is a space that provides "domains for conversations," then the disciplinary silence surrounding tradi

tional doctoral exams should be unsettling to us all (37). Applebee calls for a vi

sion of curriculum as "dynamic and changing, acquired through participation, and oriented toward the present and future rather than the past" (20). "Traditions of knowledge-in-action," he explains, "do not simply constrain us, but are open

to analysis and change" (17). But by whom? Are program faculty reluctant to

examine the exams because they simply appear to work as planned? After all, successful students continue to pass them, so the assumption may be that they

measure success. Yet, to implement reflective practice, our graduate pedagogies

should not continue tradition for tradition's sake.

Education reform is not always easy, and institutional restrictions often pro hibit local change. Even though many composition and rhetoric programs face

pressures to look legitimate and rigorous, doctoral pedagogy still retains the pos

sibility for meaningful negotiation because it is most often determined by faculty alone. As Damrosch notes, "[I]t is the genius of the university that you can

change virtually every aspect of the PhD, just so long as you still award a degree

of that name" (141). Perhaps, lurking behind the rich traditions of the PhD, scholars can find room for meaningful reform within graduate programs. As a

site, a location for change, doctoral education can be flexible, local, and seen as

less a measurement?a measuring up?than a place for renegotiating what is

considered meaningful work and useful pedagogy.

Notes

lWe would like to thank RR peer reviewers Janice Lauer and Stuart C. Brown for their helpful

commentary and suggestions, as well as Conrad Estrem and Susan Lucas for their steadfast patience,

feedback, and support. 2Used as a research project at Johns Hopkins, the dissertation was intended to motivate stu

dents to specialize, but writing a dissertation thesis was "an exercise designed more to manage stu

dents than to provide a meaningful educational experience" (Hinchey and Kimmel 90).

3During the first wave (1950-1970), in practice the "process of deciding what to teach the stu

dents belonged to educators [. . .] the process of moving students about [belonged to] testing special ists" (486). During the next two phases, however, these two roles "merge and overlap, with adminis

trators and then faculty taking on institutional and epistemological responsibilities for testing [. . .] and in the process developing a new expertise and a new discipline: writing assessment" (486).

4Beth Burmeister argues for more case studies or historical accounts of graduate programs to

augment North's description of the "fusion-based" curriculum at Albany (127).




5Perhaps in response to problems with the annotated bibliography, the program has recently

changed its examination procedures to a portfolio system. Still bearing the name "Comprehensive Examination" (a graduate school requirement), it was reconfigured for greater flexibility for students

and faculty. The two-hour oral examination remains a requirement. Students who began the program

following the change have not taken the new exams.

Works Cited

Anderson, Melissa S., and Judith P. Swazey. "Reflections on the Graduate Student Experience." The

Experience of Being in Graduate School: An Exploration. Ed. Melissa S. Anderson. San Fran

cisco: Jossey-Bass, 1998. 3-14.

Applebee, Arthur N. Curriculum as Conversation: Transforming Traditions of Teaching and Learning.

Chicago: U of Chicago P, 1996.

Beck, M., and H. S. Becker. "Modest Proposals for Graduate Programs in Sociology." The American

Sociologist 4 (1969): 227-34.

Berelson, Bernard. Graduate Education in the United States. New York: McGraw-Hill, 1960.

Bowen, William G., and Neil L. Rudenstine. In Pursuit of the PhD. Princeton: Princeton UP, 1992.

Burmeister, Beth. "Writing (into) the Academic Past, Present, and Future." Rev. of Refiguring the

PhD in English Studies, by Stephen North. Composition Studies 28.2 (2000): 113-35.

Connors, Robert J. Composition-Rhetoric: Backgrounds, Theory, and Pedagogy. Pittsburgh: U of

Pittsburgh P, 1997.

-."Dreams and Play: Historical Method and Methodology." Methods and Methodology in Com

position Research. Ed. Gesa Kirsch and Patricia A. Sullivan. 15-36.

Corbett, Edward P. J. "Teaching Composition: Where We've Been and Where We're Going." College

Composition and Communication 38.4 (1987): 444-52.

Cowen, Robert. "Comparative Perspectives on the British PhD." Working for a Doctorate: a Guide

for the Humanities and Social Sciences. Ed. Norman Graves and Ved Varma. London: Routledge, 1997. 184-99.

Damrosch, David. We Scholars: Changing the Culture of the University. Cambridge: Harvard UP, 1995.

Geiger, Roger. "Research, Graduate Education, and the Ecology of American Universities." The Eu

ropean and American University Since 1800. Ed. Sheldon Rothblatt and Bj?rn Wittrock. New

York: Cambridge UP, 1993. 234-59.

-. To Advance Knowledge: The Growth of American Research Universities, 1900-1940. New

York: Oxford UP, 1986.

Grant-Davie, Keith. "Coding Data: Issues of Validity, Reliability, and Interpretation." Methods and

Methodology in Composition Research. Ed. Gesa Kirsch and Patricia A. Sullivan. 270-86.

Hinchey, Patricia, and Isabel Kimmel. The Graduate Grind: a Critical Look at Graduate Education.

New York: Falmer, 2000.

Jako, K. L. "Let's Abolish the Ph.D. Orals." Change 6 (1974): 8.

Kirsch, Gesa, and Patricia A. Sullivan, eds. Methods and Methodology in Composition Research.

Carbondale: Southern Illinois UP, 1992.

Knoblauch, C. H., and Lil Brannon. Rhetorical Traditions and the Teaching of Writing. Portsmouth:

Boynton Cook, 1984.

Mayhew, Lewis B., and Patrick J. Ford. Reform in Graduate and Professional Education. San Fran

cisco: Jossey-Bass, 1974.

Miller, Richard E. As if Learning Mattered: Reforming Higher Education. Ithaca: Cornell UP, 1998.



416 Rhetoric Review

Nerad, Maresi, and Joseph Cerny. "From Facts to Action: Expanding the Educational Role of the

Graduate Division." Graduate Education in the United States. Ed. Maresi Nerad et al. New York:

Garland, 1997. 339-50.

North, Stephen M. Refiguring the Ph.D. in English Studies: Writing, Doctoral Education, and the

Fusion Based Curriculum. Urbana: NCTE, 2000.

Phelps, Louise Wetherbee. "Reproducing Composition and Rhetoric: The Intellectual Challenge of

Doctoral Education." Composition Studies 23.2 (1995): 115-32.

Stanford, Nevitt. "Graduate Education, Then and Now." Scholars in the Making: the Development of Graduate and Professional Students. Ed. Joseph Katz and Rodney T. Hartnett. Cambridge:

Ballinger, 1976. 243-58.

Taylor, Marcy, and Jennifer L. Holberg. "'Tales of Neglect and Sadism': Disciplinarity and the Fig

uring of the Graduate Student in Composition." College Composition and Communication 50.4

(1999): 607-25.

Welch, Nancy. "Resisting the Faith: Conversion, Resistance, and the Training of Teachers." College

English 55 (1993): 387-401.

Wolensky, Robert P. "Comprehensive Examinations and Professional Development." College Student

Journal 13.3 (1979): 287-89.

Yancey, Kathleen Blake. "Looking Back as We Look Forward: Historicizing Writing Assessment."

College Composition and Communication 50.3 (1999): 483-503.

Ziolkowski, Theodore. "The Ph.D. Squid." American Scholar 59 (1990): 177-95.

Heidi Estrem is an assistant professor of English and the assistant director of first-year writing at

Eastern Michigan University. She teaches a wide range of writing and pedagogy courses, from

first-year writing to courses for future secondary English teachers. Her research interests include

writers at varying stages of transition, particularly those entering college, those becoming teachers, and those beginning graduate school.

Brad E. Lucas is an assistant professor of English at Texas Christian University. His research focuses

on research practices in composition and rhetoric, and his other interests include oral history, inter

personal communication, visual rhetoric, and representations of trauma and crisis. He has taught courses in literary theory, humanities, technical writing, holocaust studies, and a variety of composi tion courses.



"Embedded Traditions, Uneven Reform: The Place of the Comprehensive Exam in Composition and Rhetoric...

Documents

Transcript of "Embedded Traditions, Uneven Reform: The Place of the Comprehensive Exam in Composition and Rhetoric...