Heidi Estrem Brad E. Lucas
Eastern Michigan University Texas Christian University
Embedded Traditions, Uneven Reform: The Place of the Comprehensive Exam in
Composition and Rhetoric PhD Programs
Sound doctoral pedagogy, in addition to other forms of professionalization in
PhD work, is essential in nurturing future generations of scholars in composi tion and rhetoric. Using the comprehensive exam as a focal point, this article
identifies absences and contradictions in the field's approach to evaluating the
competency of doctoral students.
I. Traditions and National Perspectives
It is a commonplace among scholars that obtaining the PhD requires surviv
ing certain rites of passage, one of which is the comprehensive exam.1 The say
ing goes something like this: You will never feel as current with scholarship in
your discipline as you did on the day of your exams. Whether or not this is true, at least that's the adage. Passing the exams does not guarantee a doctoral student
the PhD, but it marks one form of approval that "proves" graduate students are
worthy of the discipline. Within most composition and rhetoric PhD programs, assessment of student work varies: seminar papers, qualifying and/or compre
hensive exams, dissertations, and the dissertation defense. While some scholars
in composition and rhetoric have studied the general practices and procedures of
the dissertation, few details about the comprehensive exam have entered disci
plinary discussions about graduate pedagogy, institutional reform, or writing as
sessment. We hope to initiate discussions of the exam as a means of reconsider
ing the evaluation of doctoral students in composition and rhetoric graduate
programs.
Over the past 30 years, many PhD programs in composition and rhetoric
have worked within confines of local traditions tied to older literature PhD pro
grams. Some have found room for rethinking PhD requirements. With the pres sure to professionalize, many graduate students now participate in administrative
work, and written work is often directed to a larger audience than faculty readers
396 Rhetoric Review, Vol. 22, No. 4, 2003, 396^116
Copyright ? 2003, Lawrence Erlbaum Associates, Inc.
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
Embedded Traditions, Uneven Reform 397
at the department level. In many programs student work is no longer mere exer
cise or exploration: Seminar papers are configured as conference presentations
or journal articles, and the dissertation is framed as a draft of a future book. De
spite these adjustments in PhD requirements, exams for graduate students in
composition and rhetoric seem to have stayed surprisingly stable over the past 30 years. Through examining specifically the practices surrounding the compre hensive exam, we hope, as Marcy Taylor and Jennifer Holberg do, to "(dislo cate the easy narrative of professionalization" so prevalent in composition (608). The comprehensive exam is a concrete feature of doctoral programs, one that is
both intensely local and generically national at the same time?one that allows
us to pinpoint the intersections between theory and practice. In this essay we first lay out a brief sense of the historical role of compre
hensive exams within PhD programs in the United States. This history provides a contextual frame for our survey of comprehensive exam practices within com
position and rhetoric PhD programs in 2000. We highlight the programs imple
menting positive changes and discuss the tremendous difficulties inherent in
changing a practice so steeped in tradition, hierarchy, and academic rigor.
Throughout this article we do not intend to place blame on anyone, for we un
derstand the thoughtful and time-consuming work that went into establishing PhD programs in composition and rhetoric?and that continues to sustain them now. But as we continue to review discussions of the comprehensive exam, we
wonder why composition and rhetoric scholars?so well versed in writing as
sessment models and historically resistant to traditional views of students and
writing?have not done more to challenge the traditions of work within PhD
programs.
In Refiguring the Ph.D. in English Studies, Stephen North explains a
"magisterial curriculum" as the process by which potential scholars are selected
and tested to become members of an academic community. The magisterial cur
riculum?one that continues to flourish?reaches its summit with "barrier" ex
aminations that North describes as an "obstacle course" and "ritual gauntlet" for
graduate students (30-33). According to North, such examinations generally share the same characteristics:
They were offered under a variety of names?qualifying, prelimi nary, comprehensive?and assumed a variety of forms, as well. They
might, for example, have been exclusively oral, exclusively written, or (most often) some combination of the two; might have been given over several hours, or carried on for two or more days; might have been administered by a particular committee, or been open to all
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
398 Rhetoric Review
interested faculty; they might have given the candidates some choice over the "areas" to be examined, but might not, and so on. (33)
He contends that these examinations, though rigorous, did not (and do not, he
suggests) comprise the full rite of passage; instead, he asserts that the examina
tions must be validated by a faculty sponsor who will testify to the candidate's
acceptability (35). In other words, intelligence is necessary, but without the sup
port of a faculty member, a graduate student cannot be granted candidacy.
North's conclusion points to the murky relations between graduate students and
their faculty (a topic beyond the scope of this discussion), but the vagueness sur
rounding the exams prompts questions about the nature of the exam itself. How
did the comprehensive exam emerge, what is it today, and what informs its cur
rent use as an evaluative practice?
II. Increasing Evaluations and Assessments:
"Cooling Out" Structures
"All around us," writes Robert Connors, "are the data of the present, and
they constantly press on us the immediate question, 'What shall we do?'" To
gain perspective we briefly take up the "historian's questions" that Connors
cites: "'What have people done in the past?' and 'How did things come to be this
way?"' ("Dreams" 17). In the late nineteenth century, the American dissertation was established simply as a marker of difference between the BA and PhD de
grees.2 The Association of American Universities (AAU) in 1900 began to work
toward uniformity in doctorate requirements, occasionally addressing admis
sions standards (Geiger, Advance 18). In the 1920s, the number of PhDs nearly
tripled, increasing by another 50 percent in the 1930s, but there were still fewer
graduate students than most programs wanted (Geiger, "Research" 245^-6). Na
tionwide efforts at standardization were few, perhaps because of the inherent
challenges. For example, one Carnegie Foundation researcher found graduate education frustrating, confusing, and overwhelmingly difficult to describe at the
national level; he concluded that individual departments held sole responsibility for standards (249).
With rising enrollments and new programs established at other universities, schools like Harvard began restricting graduate admissions in 1930. Schools
would later curtail admissions with the help of the Graduate Record Examina
tion (GRE) established in 1937 (Geiger, "Research" 249). Whereas the disserta
tion was initially a mechanism for managing student populations, the growth of
doctoral education prompted more and various forms of evaluation. By the late
1930s, as Roger Geiger explains, "the potential university teacher was subject to
evaluative hurdles at recurrent intervals" (250). Shortly thereafter, the structure
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
Embedded Traditions, Uneven Reform 399
of PhD education became more or less consistent: "[A] student passed a compre hensive examination, acquired a reading knowledge of French and German,
handed in a satisfactory dissertation, and survived a final oral examination"
(Stanford 244). While the overall progression remains more or less intact today, the emphasis on evaluation began to shift after World War II.
The GI Bill and American competition with Soviet science prompted a sec
ond wave of university growth in the 1960s, tripling the number of PhDs earned
and leading to an increasingly impersonalized graduate school experience (Gei
ger, "Research" 253). Some of the first critiques of the exam emerged in the late
1960s but were mostly in response to administrative and programmatic ineffi
ciency?not from considering students (Beck and Becker). To streamline the in
stitutional processes for faculty and students, eventually many of the end-stage evaluations became less onerous, and many programs began to move the barrier
examinations to an earlier stage in a student's graduate career (Mayhew and
Ford 237). This, of course, took much of the pressure off the dissertation exams.
In the early 1970s, Bernard Berelson noted the lack of rigor in the final examina
tion?the dissertation defense?which had decayed into an exercise serving only a "ritualistic function" (230). He contrasted this to the "qualifying, comprehen sive examination" that had been "moved up in time" to "an earlier, less culmi
nating point" from the dissertation defense; this comprehensive exam is "not
only a real examination in substance and importance but one that contributes,
through his preparation for it, to the candidate's breadth within his own disci
pline" (199, 217). By the mid-1970s, the total number of American PhDs
awarded annually hit a plateau?approximately 34,000?that would remain somewhat consistent until the late 1980s (Geiger, "Research" 255). Calls for
abolishing the exam (Jako) were tempered by other calls to reform and redefine
the exam to reflect more realistic aims (Wolensky), but ultimately few changes were effected.
With shifts in evaluation emphases, doctoral programs not only assumed
gatekeeping mechanisms but also created attrition rates scandalously higher than
those for undergraduate programs. Robert Cowen calls the American PhD "a
highly structured experience" with "a considerable sequence of controls over the
timing and pacing of the preparation?over access to the start line"; however, "each of the quality control points means the possibility of an individual being 'cooled out' of the system in the first couple of years" (193). In one recent na
tional survey, roughly 50 percent of student respondents agreed that "evaluation
of students successfully 'weeds out' weak doctoral students" (Anderson and
Swazey 5). The process of "cooling out" or "weeding" students from programs is not the sole reason for attrition but certainly contributes to the high rates. Con
trary to the popular belief that the dissertation poses the greatest hurdle, one ma
jor study determined that "the majority of the graduate students who failed to
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
400 Rhetoric Review
earn their doctorates left the program before advancement to candidacy, not
after" (Nerad and Cerny 340). In their massive 1992 study, William Bowen and Neil Rudenstine conclude
that this middle-stage between coursework and the dissertation
has become unusually difficult. [. . . A] less well-defined interim pe riod [... that] may involve passing a general examination or qualify
ing exam and satisfying foreign language requirements, as well as
choosing a dissertation topic and, in some cases, submitting an ap
proved prospectus. (251, 254)
In fact, as one UCLA study revealed, roughly 30 percent of students leave dur
ing the early period of their doctoral studies (Nerad and Cerny 340). Other stud
ies concur, noting that roughly 25 percent of students at larger programs (10-15
percent at smaller ones) do not return for a second year, and out of those who do
return, roughly 30 percent do not reach ABD status (Bowen and Rudenstine
252-53). Of the doctoral students who do pass their exams, approximately 70
percent do not complete the degree (Ziolkowski 185). Various factors influence
the decision to leave, with 40-50 percent noting "burn-out" as the most promi nent factor in not completing their degrees (256), but one thing is clear: The
comprehensive exam not only comes at a crucial point in a student's graduate
career but is also imbued with a sense of institutional and programmatic selec
tion conjoined with high rates of attrition.
What has been noted?usually only in passing?is the lack of consistency and clarity about this middle-stage work that allows a student to cross over into
ABD status. Some researchers have noted students' lack of understanding about
the examination itself. Maresi Nerad and Joseph Cerny note that one problem is
student overpreparation for the exam:
Their ideas about the structure, scope, and standards of the exam
were vague. The aura of mystery which surrounded the exam
seemed to lead to a self-imposed perfectionism. They were disap
pointed, after taking the exam, to discover that they were given the
opportunity to demonstrate only a small part of their knowledge. [. . .] After passing their orals, they had trouble mustering enough
energy to write a dissertation proposal. (342)
If "burn-out" is the major factor contributing to attrition, then the exam may
play a larger role than has been considered in the past. Replacing the exam
with other mechanisms, however, does not necessarily ensure students will
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
Embedded Traditions, Uneven Reform 401
pass through the middle stage with less trepidation. For example, one political science program took the route of offering a formal writing option, instead of
examinations: Students submitted two papers instead of taking a preliminary examination, and as a result, the papers became "mini dissertations" in stu
dents' minds, resulting in protracted, indefinite revising and, as Bowen and
Rudenstine reveal, "it took several years, with almost total attrition, before the
situation was addressed" (266). Within composition and rhetoric scholarship, there have been a few calls to
rethink the PhD process. In Composition-Rhetoric Connors points to the exams
as a "vestigial practice" and "a last agonistic remnant" of a male-dominated aca
demic culture that perceives life as combat, an ongoing contest over power, terri
tory, and prestige: "[T]he 'defense' of the PhD dissertation or master's thesis is a
curious relic, an atrophied survival of a harder time no one remembers" (25, 47). However outdated such practices might seem, they do persist in newer forms, based now more heavily on the relationship a student fosters with her or his ad
visory committee?raising concerns that, for some students, the comprehensive
exam is more of a social contest than a measure of academic competency.
III. Assessment Traditions and Enacted Theory
Over the past 30 years, composition and rhetoric PhD programs have grown in size and number, with new programs currently in development. We wanted to
discover what vestiges of tradition were readily apparent in material descriptions of these programs, and why. To begin to draw the barest map of what underlying theories shaped comprehensive exams in composition and rhetoric programs, we
surveyed PhD program requirements from the 65 composition and rhetoric pro
grams listed in the Spring 2000 issue of Rhetoric Review. Our starting research
question was, simply: What do program documents reflect about the status of
the comprehensive exam in composition and rhetoric PhD programs? First, we
compiled exam information from each program that we could find available on
line, and we supplemented that information with print materials we had gathered for graduate advisement (information on six programs was not available). We
compiled this information into a searchable database. Keeping in mind Keith
Grant-Davie's work on the interpretive nature of coding data, we combed
through the collected materials for a cumulative sense of what the process was
like in all programs ("Coding Data"). After developing a list of key terms and
ideas, we sorted our data so that we could look for national patterns concerning the comprehensive exam.
We realize that likely not all programmatic information about the exam is
publicly available online or in other public documents, but we targeted the
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
402 Rhetoric Review
accessible documents to question the requirements and reflective practices in
programs across the country. Ideally, we would have liked an extensive survey and interview procedure of faculty and students from each program, and this
would be an excellent "next step" for scholarship in this area. For our purposes the program statements raise intriguing and unsettling questions about what has
changed?and what has not?in the ways composition and rhetoric programs
imagine the exam process today.
We understand the dangers of relying only on program documents for the
whole story, or even part of it. In an issue of Composition Studies focusing on
doctoral pedagogy, Louise Wetherbee Phelps notes:
Every doctoral program in rhetoric and composition expresses and
acts out vividly in concrete features a theory of the discipline: in its
curricular content, requirements, qualifying exams and reading lists,
range or concentration of faculty specializations, acceptable range of
dissertation topics or inquiry methods, brochures, program, and fac
ulty titles. Less explicitly (though this should be transparent to any
compositionist), its pedagogy?teaching methods, curriculum, use
of technologies, mentoring practices, assessment?embodies a phi
losophy of education, including a conception of adult learning and
adult development. (118)
Local discussions are not captured by the "concrete features" of our discipline
(the program descriptions in this case), nor are local tensions stemming from in
dividual professors or graduate students. However, for a field composed of
scholars trained in rhetoric, there are few excuses for ignoring the written em
bodiments of programmatic acts.
In most graduate programs, disciplinary experts conduct the comprehensive exam as a conventional text-performance. The exams, however, extend the tradi
tional assessment roles Kathleen Blake Yancey assigns to testing specialists and
teachers. It is useful to consider the comprehensive exam-as-assessment tool
parallel to first-year writing assessment tools that have been discussed, debated, and refined in our national journals for two decades (Yancey). Graduate educa
tors are guides, gatekeepers, dissertation advisers, seminar leaders?and often
friends and mentors ever willing to discuss, theorize, and reflect as colleagues.
Perhaps not surprisingly, given the traditional lore surrounding the comprehen sive exam, the vast majority of composition and rhetoric PhD programs have ei
ther a written or an oral comprehensive exam, and over half require both (see Ta
ble 1). Since the majority of these exams set out to test expert knowledge, it also
should be no surprise that nearly two-thirds of the programs designate faculty as
the assessors. When they do not indicate who will be assessing the exams, we
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
Embedded Traditions, Uneven Reform 403
Table 1
Administration of Comprehensive Exams
Number of
Programs (N = 59) Percent
Some kind of comprehensive exam 58 98.30
Both written and oral exams 34 57.62
Administered and Evaluated By Not specified 23 38.98
Faculty (committee, departmental, or student-chosen) 36 61.01
wonder if the lack of specification merely reflects a belief that to do so would be
stating the obvious.
Even though the field of composition now acknowledges the usefulness of
portfolios and other means of more authentic assessment of student writing, these established theories about undergraduates seem to be interpreted differ
ently for the graduate setting. After decades of concentrated scholarly effort, a
set of assumptions undergirds most forms of composition instruction and disci
plinary discussions of assessment. First-year writers often work together to draft
and redraft their essays, refining arguments and building community, and sub
mitting portfolios and writing for various audiences. In terms of evaluation, spe
cifically, scholars have developed a clearer sense of how authentic assessment
practices might work: At a minimum, purposes for the assessment should be
clearly articulated, and the assessment criteria should be provided. How, then, are these practices echoed in composition and rhetoric PhD programs?
Within individual classes and seminars, many of these pedagogical practices
likely flourish. But at the crossover point of the doctoral examination, it appears that not much has changed. Despite all we know about assessing student work, out of the 59 programs with public documentation available, only six (10 per cent) require papers, essays, or other compilations of written work (in portfolios or other configurations) to measure their students' knowledge. Only 24 (41 per cent) of these programs indicate that there is an option for students to revise their work. And no programs publicly account for collaborative work. Imagine if our conversations about undergraduate curricula across the country reflected
similar profiles. As David Damrosch explains:
If we really press the question of what would be entailed in fostering collaborative learning at the graduate level, we are likely to be led to
alter every aspect of our programs: course requirements, distribution
requirements, language requirements, and guidelines for the struc
turing of doctoral examination and of the dissertation. (158)
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
404 Rhetoric Review
Perhaps attrition rates would be lower?and completion rates higher?for doc
toral students if programs would consider what seems to be familiar pedagogy: collaborative work, revision processes, and portfolio evaluation.
Yancey's recent overview of assessment and testing gives us a picture of three
"waves" of assessment in writing.3 The most recent "third wave" of portfolio as
sessment has had the "specific effects of democratizing, localizing, and grounding
expertise of three kinds: student expertise, reader expertise, and theorist expertise"
(496). Even as portfolios unfortunately have the potential to "oppress more than
make possible," they still hold out hope within first-year writing programs in that
they encourage learners to provide themselves commentary and criticism for their
own development as writers (498). Ideally, such third-wave assessment procedures could accomplish similar goals as part of a doctoral student's progression to candi
dacy. However, documents from PhD programs suggest that an understanding of
assessment has swept through the field of first-year writing without leaving many traces in graduate pedagogy. Instead, doctoral exams remain in an assessment form
designed to examine particular disciplinary behavior.
Yancey characterizes this form of "second wave" assessment as dominated by
the holistically scored essay test, which replaced earlier tests of indirect measures
with a direct form of measurement that merely samples the target behavior, other
wise known as student writing (486). A manageable writing sample?produced under various constraints of time, place, and subject matter?becomes a way for
faculty ostensibly to measure student behavior. As Yancey notes, "The question
'How shall we evaluate writing?' concerns itself not only with methodology, but
also with behavior: which behavior should we examine?" (486). In the context of
undergraduate writing pedagogy, we often protest against high-stakes testing that
can define students entering college, overriding their previous record in high school courses. However, we have set up similar expectations for the comprehen
sive exam; as one program's documents read, "[S]tudents should also note that
good coursework grades do not automatically indicate that they are prepared to un
dertake their Comprehensive Exams." The exams, then, test something beyond what students might learn as members of a graduate program; they ostensibly test
behaviors of a soon-to-be professional. If we allow the gains from assessment the
ory to inform doctoral pedagogy, then similar questions can emerge about the prac tices used to evaluate graduate students, and the way they behave: How shall we
evaluate graduate students' writing? What behaviors are we examining, and why? While a majority of programs do not publicize their aims, nearly half (43
percent) of programs do give clear explanations for the purposes behind their
comprehensive exams. Generally, these purposes fall into three categories: criti
cal thinking abilities, expert knowledge, and purposes related to the research or
teaching abilities of graduate students (see Table 2).
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
Table 2
Stated Purposes of the Comprehensive Exam
Critical Thinking Expert Knowledge Research/Teaching Ability
Critical skills
Ability to make
connections among those areas
Ability to reason critically and creatively
Develop and defend an
informed position on
topics or questions
important to major and
minor areas of study Make a sustained critical
argument about the texts
on the reading list
Ability to criticize works and
relate them to other works
Critical ability An ability to analyze
penetratingly, persuasively, and succinctly
Ability to respond
appropriately Intellectual capacity
Ability to carry on a critical,
scholarly dialogue
Expressing it in coherent
persuasive prose
Integrating and synthesizing
large amounts of
information
The ability to make
substantive connections
among texts
Historical breadth
Appropriate knowledge of
primary and secondary works
A command of the discipline
Knowledge of pertinent
theoretical, methodological, and critical or interpretative
scholarship
Expert knowledge in areas
supported by faculty and
courses
Broad knowledge in three areas
Student's knowledge about
representative texts in
composition and rhetoric
A broad and basic knowledge A level of expertise in rhetoric
to warrant admission to
dissertation work
A mastery of the field that
clearly surpasses that of the
generalist General fundamental knowledge Student's knowledge of primary
texts, contexts, and authors
A working knowledge of
relevant secondary materials
Test overall knowledge Immersion in and conversance
with the academic topics and
literature in the fields of
study
Understanding of the works on
the reading list
Be able to rattle off definitions
of key terms
Mastery of a specialized area of
expertise Tests students' knowledge of
their lists
Student must evince a
specialized knowledge ...
not merely ready use but also
synthesis and potential for
creation
Overall knowledge
Strong familiarity with
important details of the texts
Establishes student's ability to teach and do research
(secondary to knowledge about representative texts)
To determine a candidate's
readiness to begin the
specialized work that leads
to the dissertation
Skills as a researcher,
scholar, and critic
Adequacy of training for
scholarly research
Intended to help you prepare to write your dissertation
by establishing breadth
Augments faculty adviser's
ability to certify to
potential employers the
student's teaching and
research competence
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
406 Rhetoric Review
Within the constraints of the exam, students need to show their knowledge of an increasingly unknowable field. When the field was much younger, such
feats of comprehension were somewhat manageable. However, even as early as
1984, as Edward Corbett would recall, the first edition of the Bedford Bibliogra
phy for Teachers of Writing prompted new graduate students to feel "at once in
spirited and intimidated. [. . .] New students certainly know that if they decide to
make a professional career of rhetoric and composition, they have a lot of rather
heady reading to do" ("Teaching" 447-48). Eighteen years later, not much has
changed, except that the levels of inspiration and intimidation are perhaps more
intense. As Wendy Bishop remarked during her President's address at the 2001
Conference on College Composition and Communication (CCCC) in Denver, the current generation of scholars cannot keep up with the torrent of scholarship now published in the field. Although the field's breadth defies inclusive knowl
edge, some program documents reflect a desire to have students know every
thing, reflecting a vision of an ideal writer-subject who first knows and then
writes. As one program description reads, "Remember that when you sit down to
write, the last thing you want is to have to think."
From our compilation of program descriptions, only two programs attempt to list clearly the criteria that will be used to assess graduate students. One tech
nological university states that "the quality of the student's work and perfor mance during the exams" will be based on the "quality of the arguments in each
of the four written responses" and the "quality of the thinking and knowledge demonstrated in the oral component of the examinations." Similarly, a women's
university's documents state:
To pass "with distinction," the student's performance must be judged
by a majority of the members of the committee to be exceptional in
quality and to represent a degree of knowledge regarding the areas
of examination that demonstrates, not just mastery of the subject, but
a depth and breadth of sophistication that clearly distinguishes the
performance from that of the majority of examination candidates.
While it is admirable that these two institutions have begun to consider what
they value in assessing students' work on the comprehensive exam, even these
indications give graduate students little sense of how they will know when they have met the assessment criteria. What are markers of quality in an argument? In
thinking and knowledge? In sophistication? Of the composition and rhetoric PhD programs we studied, the remaining
97 percent have no public statement about how exams will be assessed (see Ta
ble 3). Based on our experiences and those shared by others, we can guess that
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
Embedded Traditions, Uneven Reform 407
Table 3
Assessment Criteria for Traditional Comprehensive Exams
Number of
Programs (N -
59) Percent
Assessment Criteria
No criteria listed 57 96.61
Quality of arguments, quality of thinking 1 1.69
Exceptional in quality 1 1.69
Evaluation
No evaluation measures listed 41 69.49
Pass or no pass 5 8.47
Three to five ranks possible 13 22.03
Revision or retake options 24 40.67
Conditional pass with additional coursework or exams 1 1.69
Evaluators
Not specified 23 38.98
Faculty committee (doctoral committee, standing faculty
committee, or unspecified committee) 32 54.23
Student graded anonymously by faculty committee 2 3.38
Graded by anonymous faculty 1 1.69
assessment criteria for graduate students are sometimes delineated more clearly in negotiations with their committees than in program materials. But often, the
criteria do not become clear until after the exam is challenged?or over. And
while assessing graduate students' work by highly nebulous, locally determined
criteria is perhaps the inevitable result of PhD work, it also holds the negative
potential that individually determined criteria for one candidate's work could
differ radically from another's.
A longstanding rite of passage, the comprehensive exam most often posits a
student self as one that has a sense of freedom and agency, but ultimately re
mains bound by a disciplinary hegemony. Imagine, momentarily, that Yancey is
commenting on the comprehensive exam process in this passage:
the self becomes a producer?of a holistically scored essay?and thus an agent who creates text. Still, there is less agency there than it
appears. The text that is created is conventionally and substantively determined?some might say overdetermined?by an expert who
constrains what is possible [. . . T]he authorship of such a text is
likely to be a static, single-voiced self who can only anticipate and
fulfill the expert's expectations. [. ..] At best, agency is limited.
(499)
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
408 Rhetoric Review
Of these programs that indicated the process of selecting testing content, only 27
(46 percent) allow for student input, and of course, exams are most likely graded by expert faculty (although this is an assumption, since 23 programs [39 percent] do not list who will be assessing the exams, it seems safe to say that faculty do the assessment). In two programs faculty assess anonymous student work: Stu
dents' identities are somehow concealed so that a semblance of objectivity re mains. In one program the faculty are asked to work as anonymous graders (and we wonder just how anonymous they can be). Just as future scholars are about to
embark on their dissertation projects, it seems that in some instances the possi bilities for authentic agency are reduced to compliance with faculty visions of the examined students' answers to faculty-supervised questions.
The majority (70 percent) of programs report no specific evaluation options for the comprehensive exam; it is intriguing to note, however, that despite the dearth of evidence for what the exams will be assessed on, one-fifth of programs (22 percent) do delineate very specifically how students' exams will be ranked and sorted. Some designations for exams include:
No Pass, Conditional Pass, or Pass
Fail, Pass, or Pass with Distinction
Fail, Pass with Reservations, Pass, or Pass with Distinction
Fail, Low Pass, Pass, High Pass, or Honors
Failure with recommendation not to continue in program; failure with per mission to retake exam later; approval to remain in program with conditional
requirements; approval and recommendation to proceed to dissertation Letter grades
According to North, by the time students have reached PhD work in English, they have already been sorted, ranked, evaluated, and classified until only a tiny
minority of students remains (15). The comprehensive exam is then used to fur ther rank and sort the few remaining students completing graduate work, despite attrition rates that have in the past hovered near 50 percent, admissions policies that inevitably bar many interested people from continuing their graduate work, and the self-selection process that students go through before committing to doc toral work. Of course, after earning an undergraduate degree and completing years of graduate-level coursework, most students can rise to this disciplinary challenge and move forward with their careers. But as scholars and teachers, we
must ask ourselves if we are effectively evaluating our future colleagues?or if
we're simply forcing them to comply with our unreflective, unspoken expecta tions based on "what we went through" to get our PhDs.
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
Embedded Traditions, Uneven Reform 409
Almost two decades ago, C. H. Knoblauch and Lil Brannon asked us to con
sider where and how composition classrooms might reflect certain ancient be
liefs about language: "that ideas exist prior to language, that the content of a dis
course is wholly independent of its form, that knowledge is fixed and stable"
(24). They claim that often "the public oration resembled a gymnastic event, where judges know the range of acceptable behavior [.. . ] and evaluate, not
chiefly what is done, but how well a performer does what is supposed to be
done" (25). The preponderance of traditional exams reflected in our survey point to, we're afraid, a type of comprehensive exam that is in line with the "ritual
constraints" of other kinds of traditional school writing, in which each is "de
fined in its pristine generic integrity; the rules for responsible behavior are care
fully depicted; the consequences of disobedience are ominously foretold?vio
late footnote conventions and generations of scholars will groan from their
graves" (29). This set of exams, whether spread over weeks or hours, asks stu
dents to convince readers of both their general and specialized knowledge, all
through a process that does not draw on our knowledge of the dubious results
from such testing situations. These are high expectations of what is most often a
sit-down, timed essay exam (whether 3 hours, 24 hours, or weekend-long). The
performance aspect of these exams is tantamount; students are expected to show
their abilities before an audience of faculty members. Within this unique rhetori
cal situation, one that will never present itself again, doctoral students are sup
posed to show their ability to reason critically and their readiness to do further
critical work?and all expressed in coherent, persuasive prose. The expectations for graduate-level work are reasonable, but the constraints are not. After all, if
graduate faculty insist that quality work can be produced in a few hours, do they not tacitly argue that scholarship is a matter of content-transmission, rather than
careful inquiry, reflection, and socially informed textual processes?
Ultimately, our study of policy documents aims merely to begin assessing the "spirit of the enterprise" of graduate education in composition and rhetoric
PhD programs (North 194). Institutional pressures may be such that the concrete
features we have compiled above have very little relation to the actual work
within programs. The extent to which these program documents belie any at
tempt to accommodate established pedagogical assumptions may point to inertia within these programs, local exigencies, university-wide pressures for "tradi
tional" criteria, or departmental conflicts in cases where composition and litera ture coexist. We list these contingencies and hedges, knowing how true they can
be, and yet we keep returning to this unsettling question: Why, in a young and
progressive field like composition studies, do our public documents reflect a
field that is tied to highly questionable assessment practices? Of course, a new
field also needs to establish itself within the academy, and part of the way this is
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
410 Rhetoric Review
done is through assuming traditional practices. However, as our PhD programs have grown and matured, perhaps there is now room to begin rethinking what traditions we retain in doctoral education, and why.
IV. Alternative Visions of the PhD Examination
Several graduate programs are seriously reforming graduate pedagogy; the
program descriptions hint at fascinating changes that would be well worth inves
tigating in different venues. As a more detailed, public account of doctoral peda gogy, North's book has been instrumental in helping us place our discussions about graduate pedagogy in another context. While exams at SUNY-Albany are
individually flexible, "governed less by policies than by faculty practices," North
does assert the necessity of policies. They can "establish a context for those ne
gotiations: express, in principles and procedures, the agreed-upon spirit of the
enterprise" (194). In this program the "qualifying" exam is a "writing occasion"
that bridges coursework and the dissertation, serving as "a framework for shap
ing preparation [for the dissertation]" (196). The work and research students do
leading up to the exam results in a three-hour, sit-down writing "occasion" (in the placeholder of an "exam") compared to compulsories in figure skating: "[E] very one involved knows that the performer needs to cover a certain
agreed-upon ground, and do so in a way that allows her or him also to display an
acceptable range of recognizably disciplined movements" (197). Alongside this exam students also write "working papers" over the course of 6-12 months, one
of which outlines a candidate's direction for research.
North's delineation of the processes for the exam and the dissertation point us to an unasked question: Why, in a program where alternative structures are
rich with potential for meaningful alternatives, is there still a sit-down, three-hour comprehensive exam at all? The program certainly holds a lot of pos
sibility for fruitful reconsiderations of how graduate work might be more richly articulated, and we value the honesty of putting a program out for display, espe
cially after we have seen the lack of information about the workings of other
programs.4
Other program documents also hint at changes. At one university, for exam
ple, graduate students submit a Culminating Assessment that consists of either a
writing portfolio or teaching portfolio. Both options include a reflective essay, while the teaching portfolio may also include videos that reflect theory and prac tice in composition. Further, both are presented in a public setting. At another
university the documents provide a hint of intrigue; in addition to "comprehen sive written qualifying examinations," students are "further required to acquire and document a set of competencies and perspectives: historical perspective, use
of technology for scholarship and/or teaching, a research tool or method,
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
Embedded Traditions, Uneven Reform 411
interdisciplinarity in their studies, and professional experience (teaching, intern
ships, appropriate work experience)." We can presume that students will negoti
ate with committee members how to acquire and document these "further" re
quirements, but the program guidelines do not specify how this will happen.
Finally, one university has gone even further by openly abandoning the notion of a "comprehensive" exam altogether, focusing instead on a "PhD Project" that is
"an occasion to integrate various aspects of the changing disciplinary practices of English and to begin to define a broad area of inquiry for subsequent work."
The promise of this project is provocative; yet, interestingly, the exam remains in
the form of two three-hour examinations "based upon the paper and the project,"
culminating in an oral exam.
Before taking full-time positions, we were graduate student colleagues in a
doctoral program that posed an alternative to the comprehensive exam. This
Carnegie I, doctoral-granting institution had offered a composition and rhetoric PhD for almost ten years. Traditionally, its graduate students studying litera ture were given four timed, written exams, followed by an oral examination.
After a few students graduated with emphases in composition and rhetoric, the
program faculty implemented different requirements, and we were part of the first generation to face the new requirements. The newly configured compre hensive "exam" now required an extensive self-directed annotated bibliography and a publication-quality professional paper, culminating in a one-hour "dis cussion" of our material. Like so many graduate students, we went through our
obligatory grumbling and stressful moments, but for us the new process
prompted specific questions about the exam itself. When our discussions needed some grounding, we looked to our program documents for answers,
finding these requirements:
Written: Working closely with the advisory committee, the stu dent will prepare a 30-50 item annotated bibliography of major works in composition and rhetoric. This bibliography is prepared
throughout the student's time in the program. The annotations con
sist of the student's response to readings and analysis of themes and
issues, and they can often lead to ideas for further reading. The can
didate then proposes a topic growing from the reading, and, subject to advisory committee approval, writes a paper of publication qual
ity on that topic. For the additional area of expertise, the student demonstrates proficiency and competence through a written exami
nation or paper, as determined by the committee.
Oral: After the student has completed all aspects of the compre hensive written examinations, the advisory committee will conduct
an oral review of the exam, not to last more than an hour and a half.
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
412 Rhetoric Review
We understood and believed in some of the ideals behind this exam: that a
cumulative and additive project might encourage reflection during coursework
and beyond, that writing could be used to learn and interrogate that learning, and that a paper for an audience beyond our department would be a useful un
dertaking. And we valued the move toward more authentic means of assess
ment; we knew that timed essays were not fully effective for evaluating student
writers. But we also had our individual doubts about the projects we were de
veloping in lieu of the written comprehensive exam. We found ourselves en
meshed in intense discussions about assessment and evaluation, intrigued to
clarify for ourselves the purposes and aims of evaluation for doctoral candi
dacy. We realized that in a space rich in possibilities for negotiation and
change, we had been part of one program among few that were willing to re
consider the exam process directly.
This alternative structure pointedly critiques traditional exams by relying on
a very different theory of how people learn and write best, but because this alter
native still takes place within the institution, complications remain. As Richard
Miller notes, school work takes place "within the context of a deeply entrenched
bureaucratic system that exercises any number of material constraints on what
must take place in the classroom" and, of course, on what is possible in extracur
ricular work in a program. His supposition about the difference between stu
dents' and teachers' perspectives on liberatory practices unsettles us as teachers
and yet reminds us of how we felt as students: "[W]here teachers see a liberatory
practice and rising opportunities, most students see a set of requirements, an ar
bitrary system of assessment, an impediment to advancement?a bureaucracy, in
short" (19). Doctoral students in this program work with their faculty advisers to
shape their reading lists and form their scholarly essays; ultimately, however, the
faculty sits in judgment, deciding who passes and who doesn't, based on elusive or obfuscated criteria. Tensions of power and control are sidelined or approached
through the language of appropriateness rather than performing to community standards.
Paradoxically, as we worked within this revised structure for the compre hensive exam, the tensions over what was acceptable and what was not became
heightened as the nontraditional manner of these projects became clearer. Ru mors flew among graduate students about individually negotiated annotated bib
liographies of 20 pages while others' projects topped 150. Some faculty seemed
to be unsure of the purpose and parameters of this new project and used its loose
structure to formulate their own requirements for graduate students. We faced
moments of uncertainty about expectations while guidelines for the process were
still in development, but being offered an alternative has led us to think more
critically about both traditional and alternative exam structures. Like Welch we
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
Embedded Traditions, Uneven Reform 413
wonder what happens in the next, postepiphanic scene (and to the players them
selves) when we hear of alternatives implemented successfully. As graduate stu
dents, we often felt the additional burden of performing under a new, sometimes
unwritten set of rules for the comprehensive exam?even while we admired our
faculty advisers' willingness to question the vestiges under which they adminis
tered this evaluative tool.
Behind our narrative here are stories of other graduate students, stories too
fresh to be told since power relationships do not disappear with the granting of a
degree. We remember one student who left the program because of frustrations over the alternative structure. Others chose to be "grandfathered in" under older, traditional exam requirements when they saw other graduate students struggle
with this new program. We've provided these few details with some apprehen sion because they represent individual lives and struggles. And these stories are
not all ours to tell; but perhaps such confessionals are less useful than the issues
they provoke, for it is a challenge to consider the effectiveness of any type of al
ternative exam. It is especially challenging to consider the possibility that such exams can perpetuate the gray areas of the magisterial process. After all, do
alternative exams continue to foster oppression rather than reflection? Do they still encourage static, well-behaved, and singular selves who produce overdetermined, univocal texts? In an attempt to overturn "time-tested" tradition,
do they simply reconfigure tradition, redefine an emphasis on research-oriented
work, and require a new set of rituals, trials, and obstacles?
There are a variety of perspectives about the way we were examined?some
positive, others critical. The alternative exam is still a work in progress, but the
paradox of institutional reproductions abounds, preventing honest assessment:
Just as our faculty mentors struggled to implement these alternative structures within the parameters of an institutional framework, the same professional con
straints provide little room for us to reflect openly on the untold stories here. Too much is at stake among our small community; many of the dilemmas of even
speaking or writing about doctoral pedagogy, then, become once again entangled with issues of power and authority.5
V. Possibilities for Change: Continuing the Conversation
Our own experiences have drawn us to the rich site (as yet underexplored) of doctoral pedagogy. There are promising new directions for inquiry into gradu ate education, but much remains within it to be challenged by graduate students and faculty alike. Perhaps the first step is to recast examination processes into al
ternatives to traditional structures, like those described by North, hinted at in a
few other programs' documents, and undertaken by graduate students in our own
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
414 Rhetoric Review
PhD program. The next place to turn our collective gaze may be the structures
themselves. If an alternative structure still serves traditional examination and
testing roles, the effects of any newer, more theoretically sound pedagogy may be more detrimental than the processes it attempts to replace.
If we agree with Arthur Applebee that a curriculum is a space that provides "domains for conversations," then the disciplinary silence surrounding tradi
tional doctoral exams should be unsettling to us all (37). Applebee calls for a vi
sion of curriculum as "dynamic and changing, acquired through participation, and oriented toward the present and future rather than the past" (20). "Traditions of knowledge-in-action," he explains, "do not simply constrain us, but are open
to analysis and change" (17). But by whom? Are program faculty reluctant to
examine the exams because they simply appear to work as planned? After all, successful students continue to pass them, so the assumption may be that they
measure success. Yet, to implement reflective practice, our graduate pedagogies
should not continue tradition for tradition's sake.
Education reform is not always easy, and institutional restrictions often pro hibit local change. Even though many composition and rhetoric programs face
pressures to look legitimate and rigorous, doctoral pedagogy still retains the pos
sibility for meaningful negotiation because it is most often determined by faculty alone. As Damrosch notes, "[I]t is the genius of the university that you can
change virtually every aspect of the PhD, just so long as you still award a degree
of that name" (141). Perhaps, lurking behind the rich traditions of the PhD, scholars can find room for meaningful reform within graduate programs. As a
site, a location for change, doctoral education can be flexible, local, and seen as
less a measurement?a measuring up?than a place for renegotiating what is
considered meaningful work and useful pedagogy.
Notes
lWe would like to thank RR peer reviewers Janice Lauer and Stuart C. Brown for their helpful
commentary and suggestions, as well as Conrad Estrem and Susan Lucas for their steadfast patience,
feedback, and support. 2Used as a research project at Johns Hopkins, the dissertation was intended to motivate stu
dents to specialize, but writing a dissertation thesis was "an exercise designed more to manage stu
dents than to provide a meaningful educational experience" (Hinchey and Kimmel 90).
3During the first wave (1950-1970), in practice the "process of deciding what to teach the stu
dents belonged to educators [. . .] the process of moving students about [belonged to] testing special ists" (486). During the next two phases, however, these two roles "merge and overlap, with adminis
trators and then faculty taking on institutional and epistemological responsibilities for testing [. . .] and in the process developing a new expertise and a new discipline: writing assessment" (486).
4Beth Burmeister argues for more case studies or historical accounts of graduate programs to
augment North's description of the "fusion-based" curriculum at Albany (127).
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
Embedded Traditions, Uneven Reform 415
5Perhaps in response to problems with the annotated bibliography, the program has recently
changed its examination procedures to a portfolio system. Still bearing the name "Comprehensive Examination" (a graduate school requirement), it was reconfigured for greater flexibility for students
and faculty. The two-hour oral examination remains a requirement. Students who began the program
following the change have not taken the new exams.
Works Cited
Anderson, Melissa S., and Judith P. Swazey. "Reflections on the Graduate Student Experience." The
Experience of Being in Graduate School: An Exploration. Ed. Melissa S. Anderson. San Fran
cisco: Jossey-Bass, 1998. 3-14.
Applebee, Arthur N. Curriculum as Conversation: Transforming Traditions of Teaching and Learning.
Chicago: U of Chicago P, 1996.
Beck, M., and H. S. Becker. "Modest Proposals for Graduate Programs in Sociology." The American
Sociologist 4 (1969): 227-34.
Berelson, Bernard. Graduate Education in the United States. New York: McGraw-Hill, 1960.
Bowen, William G., and Neil L. Rudenstine. In Pursuit of the PhD. Princeton: Princeton UP, 1992.
Burmeister, Beth. "Writing (into) the Academic Past, Present, and Future." Rev. of Refiguring the
PhD in English Studies, by Stephen North. Composition Studies 28.2 (2000): 113-35.
Connors, Robert J. Composition-Rhetoric: Backgrounds, Theory, and Pedagogy. Pittsburgh: U of
Pittsburgh P, 1997.
-."Dreams and Play: Historical Method and Methodology." Methods and Methodology in Com
position Research. Ed. Gesa Kirsch and Patricia A. Sullivan. 15-36.
Corbett, Edward P. J. "Teaching Composition: Where We've Been and Where We're Going." College
Composition and Communication 38.4 (1987): 444-52.
Cowen, Robert. "Comparative Perspectives on the British PhD." Working for a Doctorate: a Guide
for the Humanities and Social Sciences. Ed. Norman Graves and Ved Varma. London: Routledge, 1997. 184-99.
Damrosch, David. We Scholars: Changing the Culture of the University. Cambridge: Harvard UP, 1995.
Geiger, Roger. "Research, Graduate Education, and the Ecology of American Universities." The Eu
ropean and American University Since 1800. Ed. Sheldon Rothblatt and Bj?rn Wittrock. New
York: Cambridge UP, 1993. 234-59.
-. To Advance Knowledge: The Growth of American Research Universities, 1900-1940. New
York: Oxford UP, 1986.
Grant-Davie, Keith. "Coding Data: Issues of Validity, Reliability, and Interpretation." Methods and
Methodology in Composition Research. Ed. Gesa Kirsch and Patricia A. Sullivan. 270-86.
Hinchey, Patricia, and Isabel Kimmel. The Graduate Grind: a Critical Look at Graduate Education.
New York: Falmer, 2000.
Jako, K. L. "Let's Abolish the Ph.D. Orals." Change 6 (1974): 8.
Kirsch, Gesa, and Patricia A. Sullivan, eds. Methods and Methodology in Composition Research.
Carbondale: Southern Illinois UP, 1992.
Knoblauch, C. H., and Lil Brannon. Rhetorical Traditions and the Teaching of Writing. Portsmouth:
Boynton Cook, 1984.
Mayhew, Lewis B., and Patrick J. Ford. Reform in Graduate and Professional Education. San Fran
cisco: Jossey-Bass, 1974.
Miller, Richard E. As if Learning Mattered: Reforming Higher Education. Ithaca: Cornell UP, 1998.
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
416 Rhetoric Review
Nerad, Maresi, and Joseph Cerny. "From Facts to Action: Expanding the Educational Role of the
Graduate Division." Graduate Education in the United States. Ed. Maresi Nerad et al. New York:
Garland, 1997. 339-50.
North, Stephen M. Refiguring the Ph.D. in English Studies: Writing, Doctoral Education, and the
Fusion Based Curriculum. Urbana: NCTE, 2000.
Phelps, Louise Wetherbee. "Reproducing Composition and Rhetoric: The Intellectual Challenge of
Doctoral Education." Composition Studies 23.2 (1995): 115-32.
Stanford, Nevitt. "Graduate Education, Then and Now." Scholars in the Making: the Development of Graduate and Professional Students. Ed. Joseph Katz and Rodney T. Hartnett. Cambridge:
Ballinger, 1976. 243-58.
Taylor, Marcy, and Jennifer L. Holberg. "'Tales of Neglect and Sadism': Disciplinarity and the Fig
uring of the Graduate Student in Composition." College Composition and Communication 50.4
(1999): 607-25.
Welch, Nancy. "Resisting the Faith: Conversion, Resistance, and the Training of Teachers." College
English 55 (1993): 387-401.
Wolensky, Robert P. "Comprehensive Examinations and Professional Development." College Student
Journal 13.3 (1979): 287-89.
Yancey, Kathleen Blake. "Looking Back as We Look Forward: Historicizing Writing Assessment."
College Composition and Communication 50.3 (1999): 483-503.
Ziolkowski, Theodore. "The Ph.D. Squid." American Scholar 59 (1990): 177-95.
Heidi Estrem is an assistant professor of English and the assistant director of first-year writing at
Eastern Michigan University. She teaches a wide range of writing and pedagogy courses, from
first-year writing to courses for future secondary English teachers. Her research interests include
writers at varying stages of transition, particularly those entering college, those becoming teachers, and those beginning graduate school.
Brad E. Lucas is an assistant professor of English at Texas Christian University. His research focuses
on research practices in composition and rhetoric, and his other interests include oral history, inter
personal communication, visual rhetoric, and representations of trauma and crisis. He has taught courses in literary theory, humanities, technical writing, holocaust studies, and a variety of composi tion courses.
This content downloaded from 138.237.48.248 on Mon, 2 Jun 2014 14:20:21 PMAll use subject to JSTOR Terms and Conditions
Top Related