of home economics preservice teachers in texas

D E V E L O P M E N T OF A PORTFOLIO PROTOTYPE FOR ASSESSMENT

OF HOME ECONOMICS PRESERVICE TEACHERS IN TEXAS

by

L O R A ANN NEILL, B.S.H.E., M.S.H.E.

A DISSERTATION

IN

HOME ECONOMICS EDUCATION

Submitted to the Graduate Faculty of Texas Tech University in

Partial Fulfillment of the Requirements for

the Degree of

DOCTOR OF PHILOSOPHY

Approved

\i December, 1996

< ^ ^

]\\J^^^ ^ j - '

® 1996, Lora Ann Neill

ACKNOWLEDGMENTS

The successful completion of any dissertation is more than an individual effort.

To aU those who contributed to this effort either professionally or personally, I extend

my deepest gratitude.

I wish to express my sincere appreciation to Dr. Betty L. Stout, under whose

patient guidance this professional study was conducted. Her encouragement, attention

to detail, and unique perspective were especially important during this endeavor.

Gratitude is also expressed to the members of my committee for their expertise,

encouragement, and generous commitment of time in my behalf: Dr. Ruth E. Martin,

Dr. Ginny Felstehausen, Dr. Sue Couch, Dr. Michael Mezack, and Dr. Patsy S.

Hallman, Adjunct Member and Associate Dean, CoUege of Education, Stephen F.

Austin State University, who inspired and supported my interest in portfolios and the

teaching proficiencies.

A special thank-you goes to the following: Dr. Gloria Durr, Chair, Department

of Human Sciences, Stephen F. Austin State University, and Raymond Stames,

Principal, Odessa High School, who supported my professional efforts; Doug

Hubbard, statistician, who made room in his crowded schedule for my questions; and

to Amy Wolff, Southwest Texas State University, who graciously shared her personal

teaching portfolio and her "words of wisdom" on portfolio development.

Finally, this study is dedicated to my four extraordinary children: Dirk, who

always encouraged me to follow my dreams; NathaUe, who provided support during

11

the tough times; Amy, who lovingly prodded me to get this stage of my hfe behind

me; and Paige, who counseled me, made me laugh, and refused to let me give up; and

to my two adorable grandsons: Bryson and Griffen, who give me joy and purpose.

My parents, James C. (Junmie) and Elsie Kerr, deserve a special thanks for a lifetune

of love and abundant support; and I appreciate the constant encouragement and

friendship of my sister Jane Nichols, whose hospitalhy and "Quest-ing" spfrit renews

my spirit.

ui

TABLE OF CONTENTS

ACKNOWLEDGEMENTS ii

ABSTRACT ix

LIST OF TABLES xi

LIST OF FIGURES xiii

CHAPTER

I. INTRODUCTION 1

Statement of the Problem 5

Research Questions 6

Assumptions 6

Limitations 7

Operational Definitions 8

Summary 10

n. REVIEW OF LITERATURE 12

Assessment 12

Alternative Assessment 16

Performance Assessment 20

Authentic Assessment 20

Portfolios 22

Portfolios Defmed 22

IV

Uses and Types of Portfolios 23

Portfolio Development and Disposition 25

RehabUhy and Validhy of Portfolios 28

Portfolios in Teacher Education 29

Purposes and Uses of Teaching Portfolios 30

Content of Teaching Portfolios 31

Organizing a Teaching PortfoHo 32

Research on Effectiveness of Portfolios 33

Portfolios as Assessment 35

Use of Portfolios in Hiring 36

Portfolio Development and Teaching Improvement 38

EvEiluating Portfolios 40

Summary 42

m. METHODOLOGY 44

Research Design 44

Selection of Subjects 45

The Delphi Process 48

Instrument Design 52

Round-One Instrument 52

Round-Two Instrument 58

Data Analysis 59

Summary 60

W. FINDINGS AND DISCUSSION 61

Findings of Research Questions One, Two, and Three 61

Research Question 1 63

Home Economics Preservice Teachers 64

Home Economics Teacher Educators 64

Personnel Administrators 67


Home Economics Preservice Teachers 71

Home Economics Teacher Educators 73

Personnel Administrators 73


Discussion 85




Summary 94

V. DEVELOPMENT OF A PORTFOLIO PROTOTYPE 95

Portfolio Prototype Development 96

Portfolios: An Authentic Alternative Assessment 96

The Teachmg Portfolio 97

The Selective Portfolio 97

The Reflective PortfoUo 98

VI

Evidences of Learner-centered Teaching Proficiencies . . . . 98

Portfolio Organization 98

The Portfolio Prototype 100

Prototype Organization 100

Prototype Content 101

Prototype Presentation 104

Procedures for Validation of the Portfolio Prototype 106


Discussion and Summary 115

VI. SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 117

Summary of the Study 117

Summary and hiterpretation of Fmdmgs 121

Conclusions 127

Recommendations 129

Sunrunary 130

SELECTED BIBLIOGRAPHY 132

APPENDICES

A. SAMPLE LIST AND CORRESPONDENCE 140

B. ROUND-ONE INSTRUMENT 151

C. ROUND-TWO INSTRUMENT 166

D. SELECTIONS FROM PORTFOLIO PROTOTYPE 173

E. PORTFOLIO PROTOTYPE QUESTIONNAIRE 194

vu

USEFULNESS OF EXAMPLES OF EVIDENCES IN ASSESSING MASTERY OF EACH PROFICIENCY 198

vin

ABSTRACT

In 1994, the Texas State Board of Education adopted five teaching

proficiencies for Texas educators, published as Learner-centered Schools for Texas: A

Vision of Texas Educators. The proficiencies are performance-based standards to be

used for preparing preservice teachers in Texas and address areas of knowledge,

instmction, communication, equity in excellence for all learners, and professional

development.

The purposes of this research and development study were to identify

evidences that authentically assess mastery of the five learner-centered teaching

proficiencies adopted for Texas educators and to develop a portfolio prototype, based

on the identified evidences, for assessment of home economics preservice teachers.

The evidences were selected by consensus of three stakeholder groups: home

economics preservice teachers, home economics teacher educators, and Texas public

school personnel administrators who are responsible for hiring home economics

teachers.

The content of the researcher-developed portfolio prototype was based on those

evidences of the teaching proficiencies reaching consensus in a two-round Delphi

process and on Seldin's model of portfolio organization. Evidences reaching consensus

were instmctional and evaluation materials (teacher designed); class projects related to

teaching field; creative activity; unit/lesson plans; students' evaluations; leadership and

membership in professional organizations; philosophy/goals statement; professional

IX

development activities; ability to use a variety of telecommunications; evidence of

computer skills; evidence of knowledge of models of teaching, learning styles;

evidence of attendance at Admission, Review, and Dismissal meetings or knowledge

of Individual Education Plans for special populations students; and evidence of parent

contact in student teaching. The evidences were categorized into background

information, information from others, and products of teaching. Reflective statements

were used to substantiate inclusion of the hems of evidence. The resulting portfolio

prototype was submitted to a panel of experts for evaluation of usefulness in assessing

mastery of the learner-centered teaching proficiencies.

A panel of experts rated the portfolio prototype as useful in assessing mastery

of the teaching proficiencies. Instmctional Materials (Teacher-designed) and

Evaluation Materials (Teacher-designed) rated highest for usability across all

proficiencies. Traditional evidences such as resume, transcript, and teaching

evaluations did not reach consensus for any proficiency; yet, inclusion of these items

was suggested by the panel reviewing the portfolio prototype.

LIST OF TABLES

2.1. Selected Approaches to Grouping Alternative Assessment Strategies 19

2.2. Organizational Components of Teaching Portfolios 34

3.1. Learner-centered Teaching Proficiencies Adopted in Texas 53

3.2. Summary of Responses to Round One of the Delphi Process 57

3.3. Summary of Responses to Round Two of the Delphi Process 58

4.1. Summary Responses for a Two-round Delphi Process 63

4.2. Profile of Home Economics Preservice Teacher Respondents 65

4.3. Profile of Home Economics Teacher Educator Respondents 66

4.4. Representation of Personnel Administrator Respondents According to Texas School-Size Categories 69

4.5. Profile of Personnel Administrator Respondents 70

4.6. Home Economics Preservice Teachers' Experiences With Portfolios 72

4.7. Preservice Home Economics Teachers' Ranking of Reasons for Developing a Professional Portfolio 72

4.8. Ranking by Teacher Educators as to Purposes of PortfoUos in Teacher Education Programs 73

4.9. Experiences of Personnel Administrators with Professional Portfolios 75

4.10. Personnel Administrators' Ranking of Compelling Items in the Hiring Process 76

4.11. Evidences Achieving Consensus and Failing Consensus in Round One of the Delphi Process 78

4.12. Evidences Suggested m Round One of the Delphi Process for Assessing Teaching Proficiencies 79

XI

4.13. Evidences for Teaching Proficiencies 1, 2, and 3 Retained for Round Two of the Delphi Process 80

4.14 Evidences for Proficiencies 4 and 5 Retained for Round Two of the Delphi Process 81

4.15. Evidences Achieving 70% Consensus in Round One of the Delphi Process 83

4.16. Evidences Achieving 60% Consensus in Round Two

of the Delphi Process 84

5.1. Portfolio Prototype Organization 102

5.2. Usefulness of Examples of Evidences by Proficiencies I l l

5.3. Physical/Functional Characteristics of the Portfolio Prototype 113

F.l. Usefulness of Portfolio Prototype Evidences in Assessing Mastery of Learner-centered Knowledge 199

F.2. Usefulness of Portfolio Prototype Evidences m Assessing Mastery of Learner-centered Instmction 200

F.3. Usefulness of Portfolio Prototype Evidences m Assessing Mastery of Equity in Excellence for All Students 201

F.4. Usefulness of Portfolio Prototype Evidences in Assessing Mastery of Learner-centered Communication 202

F.5. Usefulness of Portfolio Prototype Evidences m Assessing Mastery of Learner-centered Professional Development 203

xu

LIST OF HGURES

3.1. Evidences Proposed to Assess Mastery of the

Teaching Proficiencies 55

4.1. Education Service Districts in Texas 68

4.2. Evidences Not Reaching Consensus for Any Proficiency in Rounds One and Two of the Delphi Process 86

5.1. Evidences of the Teaching Proficiencies Used as a Basis for a Portfolio Prototype 100

5.2. Alignment of Prototype Evidence Examples with Teaching Proficiencies 105

5.3. Suggested Uses of Portfolios 112

xui

CHAPTER I

INTRODUCTION

School reform has assumed nationwide proportions. While many of the efforts

have centered on curricular and program reforms, "It is thne for educators and poUcy

makers in this country to see assessment as sufficiently important that it become a

focus for reform," contends Darlmg-Hammond (1990, p. 290).

One educational initiative, America 2000: An Education Strategy (Doyle,

1991) was a culmination of the efforts of the nation's govemors to adopt a set of

educational performance goals for the nation's schools that would make the U.S.

globaUy competitive. America 2000, then-President Bush's educational plan for the

nation, supported an innovative reform of education that includes restmcturing of the

following elements of the teaching/learning process: target achievements for students,

assessment and accountability, and knowledge and skills for the teaching profession.

In 1993, in the wake of national school and assessment reform movements,

Texas conducted a statewide series of focus groups to discuss proficiencies that

educators of the future, both teachers and administrators, should demonstrate. The

Commission on Standards for the Teaching Profession (CSTP) and the Consortium of

State Organizations for Teacher Education (CSOTE) co-sponsored the focus groups.

The five learner-centered proficiencies for teachers, developed through a

consensus process and adopted by the Texas State Board of Education, represent

content, pedagogy, and the ability to put theory into practice. The set of proficiencies,

published as Learner-Centered Schools for Texas: A Vision of Texas Educators

(1994), are performance-based standards for Texas educators and serve as an

accountabihty system for educator preparation programs in Texas. Of concern to this

study is the preparation of home economics preservice teachers and the effectiveness

of home economics teacher education programs based on the leamer-centered

proficiencies.

The five leamer-centered proficiencies for teachers in Texas are as follows:

1. The teacher possesses and draws on a rich knowledge base of content,

pedagogy, and technology to provide relevant and meaningful learning

experiences for all students (p. 3).

2. To create a leamer-centered community, the teacher collaboratively

identifies needs and plans, implements, and assesses instmction using

technology and other resources (p. 4).

3. The teacher responds appropriately to diverse groups of learners (p. 5).

4. While acting as an advocate for all students and the school, the teacher

demonstrates effective professional and interpersonal communication

skills (p. 6).

5. The teacher, as a reflective practhioner dedicated to all students'

success, demonstrates a commitment to learn, to hnprove the profession,

and to maintain professional ethics and personal integrity (p. 7),

Assessment of the effectiveness of Texas teacher education programs will be

based on the five leamer-centered teaching proficiencies, according to Learner-

Centered Schools for Texas (1994). Evidences to assess mastery must authentically

reflect the proficiencies to which they are linked. Assessment must be tied to

instmction, improvement of practice, and the creation of greater knowledge

(Lieberman, 1991). The five teaching proficiencies serve as the performance criteria

for Texas teachers and teacher preparation programs.

A review of the Iherature reveals a trend toward alternative forms of

assessment, often referred to as performance assessment or authentic assessment.

Performance assessment involves students in performing tasks that reflect "real-life"

situations and challenges students to demonstrate what they have learned by producing

a product or performance. It requires students to use prior knowledge, recent learning,

and relevant skills to demonstrate proficiencies (Stiggins & Conklin, 1992).

Authentic assessment is any means of assessment that assesses performance or

knowledge within a context of real-life situations (Arter, 1993; Meyer, 1992) and

allows the student to be an active participant in the assessment process (Grady, 1992).

Arter (1993) asserts that an activity is not "assessment" if performance criteria are not

established or if there is no way to judge, evaluate, or critique the responses.

To provide evidence of quality teaching correlated to each of the five

proficiencies for educators in Texas, the Institute for Performance-based Accountability

(IPBA, 1994) identified portfolios as one means of assessing mastery of the

proficiencies. While portfolios are generally considered an authentic assessment,

Mhchell (1992) maintains that portfolios are the most widely used form of

performance assessment.

The portfolio, as an assessment strategy, is a shift from the more tradhional

summative assessment, characterized by scores on a standardized test. French (1992)

notes that ". . . portfolios tend to blur the critical Imes between instmction and

assessment and between formative and summative assessment . . . " (p. 258).

Ashehnan and Lenhoff (1994) assert that portfolios ". . . represent tiie mtersection of

assessment and instmction and . . . provide a framework for viewing evaluation as a

complex and multidimensional dynamic . . . " (p. 65).

A teaching portfolio is a tool for communicating teaching accomplishments and

represents a complete picture of students' learning and growth (Grady, 1992; Arter,

1993; Wolf, LeMahieu, & Eresh, 1992). The teaching portfoHo is a tangible record of

experience that shows progress from the beginning of the preservice program through

student teaching and beyond and is effective in assessing student progress (IPBA,

1994; Cramer, 1993; Reilly et al., 1993). Portfolios are useful to administrators in

evaluating a teacher candidate's credentials and teaching proficiency (Bouas, Bush, &

Fero, 1994).

Obtaining basehne data about student performance is critical to any assessment

model. Portfolios offer one means of ". . . capturing baseline data and adding data

over thne which can clearly show gain or change" (French, 1992, p. 260).

ff home economics teacher education programs are to be accountable for

student learning, home economics teacher educators must be mvolved in the

development of the assessment methods to measure what home economics preservice

teachers know and can do. Home economics teacher educators play a key role in

guidmg student development of portfolios that authentically represent mastery of the

five leamer-centered teaching proficiencies outlmed in Learner-Centered Schools for

Texas (1994).

The teaching portfolio can demonstrate the extent to which home economics

preservice teachers meet the leamer-centered proficiencies established for Texas

educators. A review of the literature reveals no research on utilizing portfolios as a

means of assessing performance of home economics preservice teachers.

Statement of the Problem


evidences that authentically assess mastery of the five leamer-centered teaching



The evidences of the proficiencies are to be selected by consensus of three stakeholder

groups: home economics preservice teachers, home economics teacher educators, and

Texas public school personnel administrators who are responsible for hiring home

economics teachers. The five proficiencies outlined in Learner-Centered Schools for

Texas (1994) are performance-based standards for preparing educators and serve as an

accountabihty system for educator preparation programs in Texas.

Research Questions

The study was guided by the following questions:

1. What are selected characteristics of this study's participant groups: home

economics teacher educators, home economics preservice teachers, and Texas

public school personnel administrators responsible for hiring home economics

teachers?

2. How are portfolios currently being used by home economics teacher educators,

home economics preservice teachers, and Texas public school personnel

administrators who are responsible for hiring home economics teachers?

3. What types of portfolio evidences, based on the leamer-centered teaching

proficiencies, authentically assess the performance of home economics

preservice teachers?

4. Will a panel of experts judge a portfolio prototype - based on evidences

selected by home economics teacher educators, home economics preservice

teachers, and Texas public school personnel administrators responsible for

hiring home economics teachers - useful for assessing mastery of the five

leamer-centered teaching proficiencies?

Assumptions

The foUowing assumptions guided the development of the research:

1. The proficiencies outlined in Learner-Centered School for Texas, adopted by

the Texas State Board of Education m 1994, guide home economics teacher

education programs in Texas and wUl be used to assess teacher proficiency in

Texas.

The home economics teacher educators, home economics preservice teachers,

and Texas public school personnel administrators surveyed in this study are

experts in their fields; and their selections of and suggestions for evidences

represent authentic assessments for the leamer-centered teaching proficiencies.

Consensus can be reached in selecting evidences of the five leamer-centered

teaching proficiencies resulting in a portfolio prototype useful to home

economics teacher educators and Texas public school personnel administrators

responsible for hiring home economics teachers.

The researcher-developed instruments will secure data adequate for analysis

and for developing a portfolio prototype.

Limitations

To narrow the scope of this study, the following limitations were proposed:

1. The study was limited to the development of a portfolio prototype and

did not attempt to develop criteria for evaluation of portfolios by

individual teacher education programs.

2. Comparisons were made between the survey responses and the review

of literature to determine the appropriateness of the evidences provided

by survey responses for inclusion in a portfolio.

3. The survey was Ihnited to home economics teacher educators m Texas

in Spring 1995 semester, home economics preservice teachers during the

Spruig 1995 semester, and to Texas public school personnel

administrators responsible for hiring home economics teachers selected

m a random samplmg of Texas public school districts.

4. The study was limited to portfolios as only one of a number of means

of authentic assessment.

5. The study did not attempt to assess subject-matter competencies.

Operational Definitions

The following terms were given expHch, operational definitions for use in this

study.

Assessment - Assessment is the process of collecting or organizing information

and data m ways that make it possible for the data to be evaluated (Chittenden, 1991).

Alternative Assessment - Alternative assessment (often known by other terms

such as "authentic assessment," "naturalistic assessment," or "performance

assessment") is any means of assessment that is different from traditional assessment

methods such as standardized tests, multiple-choice tests, and pencil-and-paper tests.

Authentic Assessment - Authentic assessment is any means of assessment that

assesses tasks, skills, or knowledge within a context of real-life situations (Arter, 1993;

Meyer, 1992); requires higher-order thinking (Herman et al., 1992); and/or allows the

student to be an active participant in the assessment process (Grady, 1992).

8

Evidences - Data that provide proof of mastery of each of the five leamer-

centered teaching proficiencies.

Performance Assessment - Performance assessment is any means of authentic

assessment, usually in the form of a task (Raybom, 1993), that requues a smdent to

generate a response rather then choose a response (Wiggins, 1992).

Personnel Administrators - Public school administrators who are responsible for

hiring secondary home economics teachers in a Texas pubhc school district, mcluding

high school principals, superintendents, personnel directors, assistant superintendents

for secondary personnel, and executive directors.

Portfolio - A portfolio is a student-selected (Courts & Mclnemey, 1993)

collection of evidences that demonstrates mastery of a body of knowledge or of a set

of skills over a period of time (Feuer & Fulton, 1992; Herman et al., 1992). A

teaching portfolio includes evidences of teaching (Seldin, 1993).

Preservice Teachers - Preservice teachers are those students enrolled in teacher

education programs who are studying to become home economics teachers or who are

returning to achieve teacher certification through post-baccalaureate or Jamison Bill

programs. Post-baccalaureate students hold one degree in a home economics field and

return to take a block of education courses and complete student teaching or an

internship. Students with a degree in home economics who have been out of school

for at least three years and retum for teacher certification may enter under the Jamison

Bill which aUows credit for work experience in the field of theh- teaching certificate.

Prototype - A prototype is an initial functioning model of a proposed design

(Webster's Nmth New CoUegiate Dictionary, 1989).

Teacher Proficiencies - Teacher proficiencies are those five standards and theu

descriptive statements which were adopted by the Texas State Board of Education and

published in Learner-Centered School for Texas m January 1994. Teaching

proficiencies, leamer-centered proficiencies, and proficiencies will be used

interchangeably in this study.

Summary

Leamer-centered teaching proficiencies are vital components of home

economics teacher education programs in Texas. Evidences that authentically assess

mastery of the proficiencies are important to the teacher assessment process. Texas

home economics teacher education programs have the opportunity and responsibUity to

initiate alternative assessment procedures based on the five leamer-centered teaching

proficiencies.

Three groups of stakeholders, concerned with quality education in Texas, were

identified for this study: home economics preservice teachers, home economics

teacher educators, and Texas public school personnel administrators who are

responsible for hiring home economics teachers. Home economics teacher educators

are key to guiding preservice home economics teachers in demonstrating mastery of

the teaching proficiencies. Personnel administrators are concerned with assessment of

10

home economics teachers in the hiring process and in classroom assessment after

hiring.

A student-developed portfolio is one method for home economics teacher

educators, preservice teachers, and personnel administrators to use in assessing mastery

of the proficiencies. A portfolio prototype could serve as a model for home

economics teacher educators to use when guiding students in developing portfolios that

authentically represent mastery of the teaching proficiencies.

This chapter mcluded a statement of the problems for this study: to identify




The statement of the problems guided the development of four research questions on

which this study is based. Assumptions, limitations, and operational definitions that

guide this study were provided.

11

CHAPTER n

REVIEW OF LITERATURE





The evidences of the proficiencies were selected by consensus of three stakeholder


Texas public school personnel administrators who are responsible for hiring home




The review of literature includes a brief synopsis of alternative, authentic, and

performance assessment as a guide in estabUshing a basis for portfolios as an

alternative assessment tool for preservice home economics teachers. Research studies

on portfolios as an alternative assessment tool, use of portfoHos in teacher education,

and portfolio effectiveness are reviewed in greater depth.

Assessment

In a review of hterature on the topic of assessment, the following issues

emerged: accountability, differences between assessment and evaluation, learning

12

theory as related to assessment, and dissatisfaction with traditional assessment. One

fauly constant theme in the literature on assessment and the reexamination of the

teaching and learning relationship is accountability (Stiggins & Conklin, 1992).

Schools and educators are accountable for students' attainment of educational

outcomes. While nationwide school reform has centered on curricular and program

reforms, many educators, including Darling-Hammond (1990), contend that assessment

is the place to begin educational reform.

Although assessment and evaluation often are spoken of synonymously, there

are important differences between the two. Assessment, according to The

Encyclopedia of Educational Testing, is a process of ". . . collecting or organizing

information and data in ways that make it possible for people to 'judge' or evaluate

[the data]" (Chittenden, 1991, p. 26). Assessment requires ". . . multiple indicators

and sources of evidence . . . " (p. 26).

Evaluation, on the other hand, is the process of judging the value or worth of

tiie assessment data (Chittenden, 1991; Arter, 1993) or of quantifymg the performance

(Marzano, 1992). The judgment process results m, for example, the assignmg of

scores or grades; hiring, promoting, or fuing of personnel; or granting of certification

or tenure.

Educators are advocating a clearer relationship between teaching and

assessment of leammg (Darlmg-Hanunond, 1991; Marzano, 1992; Lieberman, 1991;

Chittenden, 1991). Herman et al. (1992) noted a move from wide use of behavioral

abstract theory to increased use of cognitive theory in learning and assessment.

13

Sunmons (1994) maintained that assessment is an essential component that guides the

process of learning. As schools move toward curricula that focus on thinking

processes, integrated skills, and real-life apphcation of knowledge and skills, the

methods of assessment must also change.

"Good assessment is built on current theories of learning and cognition and

grounded in views of what skills and capacities students will need for future success"

(Herman et al., 1992, p. 75). "Chizens m the 21st century will not be judged by their

ability to bubble in answers on test forms" (Hiebert & Calfer, 1989, p. 53).

Research on learning suggests that testing procedures (i.e., criterion-referenced

and standardized tests) ". . . fail to measure students' higher-order cognitive abUities or

to support then capachies to perform real-world tasks" (Darling-Hammond, 1991, pp.

220-221). Marzano (1992) stated that standardized tests ". . . measure students' ability

to acquire and integrate information, not to extend and refine it or to use it

meaningfully" (p. 171).

Wolf, LeMahieu, and Eresh (1992) pointed out that assessment has too often

been driven by " . . . concems for measuring and reporting achievement data for

outside audiences" (p. 10). Chittenden (1991) reported the current need to organize

the assessment data m ways that are " . . . credible and comprehensible to all

constituencies: student, teacher, parent, and conununity . . . " (p. 23). Accountabihty

is vital to meaningful assessment. Meaningful assessment must accurately reflect the

criteria h is measuring.

14

Dissatisfaction with standardized achievement tests, criterion-referenced tests,

and teacher-made multiple-choice tests has led to exploration of alternative methods

(Popham, 1993; Raybom, 1993). Critics che "pencil-paper" methods of assessment as

seldom having any relationship with real-world uses of the knowledge being measured.

Such methods of assessment evaluate skills in isolation, limit thinking processes, and

ignore knowledge that students bring to the learning situation (Darling-Hammond,

1990; Lieberman, 1991; Zolhnan & Jones, 1994). "Typical tests overassess knowledge

and underassess 'know-how with knowledge'" (Wiggins, 1992, p. 27).

Grady (1992) asserted that there is little correlation between testing and

producing successful students and that standardized tests do not provide useful

information for making teaching/learning decisions. Standardized tests, according to

Mitchell (1992), lack validity and reliability for classroom purposes because the tests

may cover material that is not taught in the classroom. Darling-Hammond (1990)

contended that tests are ovemsed and misused in decision-making about students.

Haney (1991) maintained that tests rarely help students learn and noted that

"highstakes testing" usuaUy narrows the curriculum as teachers "teach to the

[standardized] test" (also Herman, 1992).

The standardized test has shaped and directed curriculum and teaching for the

past four decades. For teacher educators, rethinking assessment may be the place to

begin shaping and directing the curriculum and teaching of the next decades.

15

Alternative Assessment

Altemative assessment is a common theme in the literature on assessment

reform. Altemative assessment is usually described as any means of assessment

different from traditional methods, i.e., standardized tests and pencil-and-paper tests.

Trends, benefits, characteristics, and challenges of altemative assessment are described.

Two types of altemative assessment, performance and authentic, are explored.

Assessment measures, in general, help educators set standards about what is

important, what deserves focus, and what teachers expect as good performance; create

instmction pathways to meet the needs of the students; motivate student performance;

provide diagnostic feedback for student groupings or instmctional needs; assess or

evaluate progress of individual students, of program effectiveness, or of areas of

strength or weakness in the curriculum; and communicate progress to others (Herman

et al., 1992). The goals of ahemative assessment are shmlar: to capitalize on the

actual work of the classroom, to increase student involvement in evaluation, and to

meet accountabihty concems.

Trends in altemative assessment range from single attribute assessment (i.e.,

knowledge or recall only) to muhidimensional assessment strategies that recognize

diverse abiUties and muhiple mteUigences (Marzano, Pickering, & McTighe, 1993).

Individual assessment has been expanded mto collaborative products that showcase

group process skills (Herman et al., 1992).

Ahemative assessment provides a clearer relationship among teaching, learning,

and assessment and is more consistent with current educational research. The

16

emphasis is on formative assessment rather than strictly summative assessment

(Association for Supervision and Curriculum Development [ASCD], 1991).

Proponents argue that altemative assessment benefits the teaching/leaming/assessment

process by capturing a richer array of what students know and can do, portraying the

process(es) by which students work, aligning assessments with important and

meaningful outcomes, providing continuous and ongoing feedback, integrating

assignments with instmction, and putting teachers back into the center of the

assessment process (O'Neil, 1992; Wortiien, 1993; Herman et al., 1992; Arter, 1993).

Common characteristics of altemative assessment provided by Herman et al.

(1992) are as follows:

1. Ask students to perform, create, produce or do something.

2. Tap higher-level thinking and problem solving.

3. Use tasks that represent meaningful instmctional activities.

4. Invoke real-world applications.

5. People, not machines, do the scoring, using human judgment.

6. Require new instmctional and assessment roles for teachers.

Assessment forms that allow students to apply knowledge m practical, real-life

ways include open-ended questions, problem-solving, hands-on experiments,

investigations, laboratories, oral discourse, computer shnulations, performances,

demonstrations, essays, exhibhs, reflections/journals, and portfolios (Herman, 1992;

Lieberman, 1991; Feuer & Fuhon, 1992). Ahemative assessment provides a vehicle

17

for interaction with students through ongoing discussions, reflections, and student-

generated mbrics.

Altemative assessments are viewed as any strategy different from the

traditional paper and pencil tests such as performances, products, and portfolios. The

review of Iherature revealed a number of ways of grouping ahemative assessment

strategies. For example, Worthen (1993) grouped ahemative assessment strategies m

three categories: process, products, and performances. From the review of hterature,

the researcher developed a table showing four authors' approaches to grouping

altemative assessment strategies (see Table 2.1).

All four authors included products as a part of the total assessment plan.

Portfolios appear as a separate category on two of the lists. Arter (1993), however,

considered portfolios as a subcategory of products. Performances in two of the lists

and behaviors in the other two appear to refer to a demonstrated action; whether that

action follows a prescribed task or is spontaneous is not obvious. Communication and

interaction appear on two of the Usts. Whether considered separate from products or

as a subcategory of products, portfolios are recognized as an altemative assessment

strategy.

18

Table 2.1 Selected Approaches to Grouping Altemative Assessment Strategies.

Aut

hors

ra

tegy

ro

ups

^ o

Worthen (1993)

Process

Products

Performances

ASCD (1991)

Behavior

Products

PortfoUos

Personal communication

Betts (1994)

Products

Performances

PortfoUos

Alter (1993)

Student behaviors

Products

Classroom interaction

Altemative assessment entails new roles for teachers and students and requires

a strong commitment to increased involvement in the processes of assessment and

evaluation of teaching and learning in classroom. The ASCD (1991) contended that

students should be actively involved in assessing their own work.

If students are to be effective in the new assessments, teachers will need to

revise course assignments and activities to support the new assessment strategies.

Devising altemative assessment strategies, involving students m the collaborative

development and assessment of their work, and assessing student performance and

learning will take tune and energy (Popham, 1992; Wigghis, 1992).

While Worthen (1993) cautioned that altemative assessment is not a panacea to

all assessment or education ills, Lieberman (1991) asserted that altemative assessment

can be a powerful force for ". . . raismg standards, reforaung schools, and retiimkmg

American education" (p. 220). "Educators should help to shape the future dnection of

altemative assessment by gaining experience with it and by continuing only those

practices that work well" (Worthen, 1993, p. 454).

19

Performance Assessment

Altemative assessment is known by many popular nom de plumes including

performance, and authentic, as well as direct, extended, different, and naturalistic

(Perrone, 1991; Herman et al., 1992; Worthen, 1993). While these terms are often

used interchangeably, some writers more narrowly defme them.

Performance assessment reflects real-life situations and challenges students to

demonstrate what they have learned. Students are required to generate or create an

answer (Wiggins, 1992) or perform a series of tasks (Raybom, 1993) rather than select

an "A-B-C-D" response.

Herman et al. (1992) used the term performance-based assessment when

referring to any altemative assessment measure. Raybom (1993) used the term

"hands-on assessment" to refer to performance assessment, while Perrone (1991)

referred to performance assessment as "exhibits of learning."

Fisher (1993) poshed that students will achieve more if they are asked to

produce sometiiing, not just select an answer; and they and their teachers wUl fmd the

process more mterestmg. Wiggms (1992) equated performance assessment with

synthesis, not just sunplistic tasks but " . . . puttmg h all together . . . with good

judgment" (p. 28).

Authentic Assessment

While performance assessment requues the student to demonstrate a desned

behavior that reflects real-life situations, authentic assessment requnes that the student

20

demonstrate a desired behavior in a real-life context. Arter (1993) concurred that for a

performance task to be "authentic" the task must elicit the behavior that the assessor

wants to see and must provide real-life circumstances. "The aim of authentic

assessment is to engage students in challenges that better represent what they are

likely to face as professionals and as responsible citizens" (Wiggins, 1992, p. 23).

For an assessment strategy to be called authentic, the question must be asked

"Autiientic to what?" (Meyer, 1992; Wigghis, 1992). To be considered authentic, the

assessment must have meaning to the student (i.e., can be related to a clearly defmed

objective or to an experience with which the student is familiar), be rooted in real-life

situations (Meyer, 1992; Arter, 1993), utiUze a wide array of lifelike performance

tasks (Popham, 1993), and aUow the student to be involved in the assessment process

(Grady, 1992). Meyer (1992) further asserted that to tmly be an authentic assessment,

the strategy must put the student in control of tuning, topic, pacing, and conditions.

Authentic assessment may include a wide variety of assessment strategies including

both performance tasks and portfolios.

Worthen's definition of altemative assessment, ". . . dnect examination of

student performance on significant tasks relevant to life outside of school" (1993, p.

445), illustrated the relationship of autiientic and performance assessment. Valencia,

Hiebert, and Afflerbach (1994) contended that authentic assessment is ongomg,

captures the complexhy of the task, and chronicles student development by considering

the full range of relevant experiences and accomplishments as multiple indicators of

21

achievement. Herman (1992) maintained that when assessments model authentic skill,

performances of teachers, as well as students, improve.

It is clear from the review of Iherature that performance and authentic

assessment are types of altemative assessment consistent with ongomg educational

research on developing a clearer relationship among teaching, learning, and

assessment. Benefits to students and teachers are evident. Altemative assessments

will require greater involvement of both groups in the assessment process.

Portfolios

Portfolios are one of a variety of altemative assessment methods requiring

increased smdent involvement. The review of literature provided a composite

definition of portfolios, outlined the uses and types of portfolios, revealed factors to

consider when developing portfolios, and broached the issue of validity and reliabilhy.

Portfolios Defined

A portfolio is a focused, systematic collection of papers, works, and artifacts

about a person, project, and/or topic (Feuer & Fulton, 1992; Paulson & Paulson, 1990;

Valencia et al., 1994). A portfolio shows what a student can do over a course of thne

rather than what a student can do on a particular day or hour (Feuer & Fuhon, 1992;

Herman et al., 1992; Raybom, 1993; Touzel, 1993).

Portfolios are generally considered, along witii performance tasks, as one type

of autiientic assessment (Betts, 1994; Worthen, 1993). Otiier writers, however,

22

categorize portfolios as a type of performance assessment (Feuer & Fulton, 1992;

Mitchell, 1992; French, 1992). Mitchell maintains that no other form of performance

assessment is as widely used.

Uses and Types of Portfolios

Portfolios, long used in areas such as art, architecture, and journalism, are

being used from kindergarten through university for assessment or evaluation of

learning, grading or promotion to the next grade, self-assessment, teaching

enhancement, and hiring and promotion decisions (Seldin, 1993). Portfolios for

assessment purposes are most frequently used in curriculum areas such as reading,

writing, and math. Portfolio use is increasing in areas such as science, computers, and

social studies.

A fast-moving national practice is the use of portfolios by university facuhy to

achieve merit, promotion, or tenure and to enhance teaching. Retuing university

faculty may leave their portfolios as a legacy and model (Seldm, 1993).

The portfolio, as an assessment strategy, is a shift from the more tradhional

summative assessment, characterized by scores on a standardized test. French (1992)

noted that ". . . portfolios tend to blur tiie critical lines between mstmction and

assessment and between formative and summative assessment . . . " (p. 258).

Ashehnan and Lenhoff (1994) asserted tiiat portfolios ". . . represent tiie mtersection of

assessment and instmction and . . . provide a framework for viewing evaluation as a

complex and multidhnensional dynamic . . ." (p. 65).

23

Courts and Mclnemey (1993) categorized portfolios as two types:

nonselective, a chronological collection of all of a student's work in a course or over a

series of courses; and selective, a selection of hems from a larger body of collected

work. Obvious disadvantages of nonselective portfolios are the sheer amount of

materials stored and the time it will take for the reviewer to view the portfolio. The

selective portfolio, on the other hand, helps students focus on significant and

meaningful aspects of learning, reflect on the relevance of theu learning, and make

connections between learning and real-life.

Portfolios exhibh the student's efforts, progress, and achievements in one or

more areas (Paulson, Paulson, & Meyer, 1991). The work collected in a portfolio can

provide information about the student's attitude and motivation (Paulson & Paulson,

1990), record the evolution of thinking (Wolf, 1989), and compose a portrait of the

student as an able learner (Wolf, LeMahieu, & Eresh, 1992).

The use of portfolio-based assessment is supported by cognitive learning

theory. Grady (1992) pointed out that portfolios accommodate a divershy in learning

styles, pacmg, and strengths. Touzel (1993) maintained that using portfoHos with

students results in higher-order thinking and creativity.

Arter and Spandel (1992) described several factors that mfluence the use of

portfolios such as the purpose, the focus, the audience, the evidence, the thnelme, and

the fmal disposition of the portfolio. Portfolios should have a defmed purpose with

pre-established standards or criteria (Grady 1992; Cramer, 1993). Examples of

questions to be answered before portfolio development begins are Is the portfolio an

24

add-on type or wUl it be continually evaluated and redeveloped? How long wiU items

stay in the portfolio? Where wUl the portfolio be stored?

An important factor that relates to the purpose is whether or not the portfolio

will be used for assessment, evaluation, or both. If the portfolio is used for

assessment purposes only, timelines and deadlines should be set for assessment.

Criteria for evaluation should be clearly defmed before beginning the portfoHo

development. Students need a clear understanding of what a portfoHo is and how they

wiU be expected to develop one.

PortfoHo Development and Disposition

PortfoHos represent highHghts or milestones of learning. Simmons and Resnick

(1993) compared portfoHos with the accumulation and display of scouting badges

acquired over a number of years. A portfoHo should represent the student's best work

and should be designed to ". . . prove a particular point" (Wiggins, 1992, p. 23).

The stmcture of the portfolio is coUaboratively defined by the teacher/mentor

and the student in the context of the goals and objectives established to reach the

purpose. The portfoHo owner decides which materials will provide the most useful

evidence of competence, thereby actively involving the learner in the process (Grady,

1992).

The evidence included in the portfolio must support the purpose. Shackelford

(1993) pointed out that who will read the portfoHo (audience), why they will read h

(purpose), and in what mode they wiU read it (print, hnage, video, audio) need to be

25

established before beginning the portfolio development process. If the portfolio is for

self-improvement, the creator may be the only reader; if the portfolio is for a grade,

the portfolio will be read by others and wiU need greater clarity.

A key factor in portfolio development is that as students create a portfolio, they

must analyze or evaluate their own work. Howard Gardner contended that portfolios

can ". . . assess a student's abiHty to produce, perceive, and reflect" (ched in Brandt,

1990, p. 35). This reflection process includes focusing on why an item was included

in the portfoHo, how the piece illustrated a selected criterion, or what the selected

piece meant to the professional or personal development of the creator.

French (1992) suggested that portfolio development include measurement of the

foUowing student outcomes: academic outcomes, attitudinal changes, and inteUecmal

growth. PortfoHos show what students can produce and how their perceptions and

feelings affect what and how they produce (reflections). Students' reflections about

their own work, progress, and growth can be either written or verbal.

Brief reflective comments attached to evidences of learning often are referred

to as "captions." One or more reflective pieces may be included in the portfolio

(Polin, 1993; Wolf, 1989) or each entry may have a reflection or evaluation comment

attached (Feuer & Fulton, 1992). Accordhig to Green and Smyser (1996), reflection is

key to the teaching portfoHo; h aUows the writer to weave together all parts of the

complex teaching experience.

Through the reflective processes used in developing portfoHos, students are

exposed to metacognition: the process of thinking about their thinking processes. If

26

students are to become independent leamers, they must not only know something but

they must also be aware of how they know it and how it relates to the course content

or goals (Vacca & Vacca, 1989; Angelo & Cross, 1993). Students must recognize

how content, performance, and self mteract to affect learning. The written reflections

promote higher levels of cognitive engagement, more evaluative thinking, and a

greater mclmation to revise poshions (Rousculp & Maring, 1992).

Portfolios are both a product and a process. Shackelford (1993) contended that

the portfolio may actually be second to the process. Wigghis (1992) concurred that

the focus should be less on the finished product and more on the student process.

Students and their mentors must value the process of portfolio development in order to

commit to the time involved in developing a portfolio.

The final disposition of the portfolio should be considered in the portfolio

development process. While most authors feel strongly that the portfolio belongs to

the creator of that portfolio, student portfolios may be used as models for the next

group of students, as study cases for preservice teachers, and as evidence of program

effectiveness (Wolf, 1989; Rousculp & Maring, 1992; Reilly, HuU, & Greenleaf, 1993;

Seldm, 1993).

Portfolios build a community of leamers, aUowing stakeholders (teachers and

students) to share ideas (Rich, 1994). Students facilitate theu own and others' learning

by sharing with others, offering support, suggesting possibilities, and evaluating ideas

in a social context. Important learning advantages of portfolios are feedback from

mentors and the discovery of new strategies when reviewing the work of others

27

(Rousculp & Maring, 1992). The strong sense of ownership in the development

process builds enthusiasm for the learning/creating process.

ReUabUhy and Validhy of Portfolios

The reliabilhy and validity of portfoHos are questions that must be answered.

Haney (1991) noted that assessments must be validated for then intended use;

therefore, the purpose of the portfolio hnpacts hs validhy and reliabilhy. The portfolio

is most often used as a formative assessment measure. French (1992) maintained that

while summative assessment requires validity and reliabiHty, formative assessment,

used to guide student development, does not. The question of validhy and reHabilhy

of a particular portfoHo would depend on its specified purpose.

Authentic assessments achieve validity and reHabiHty by emphasizing and

standardizing the appropriate criteria for scoring (Betts, 1994). "The vary [sic] nature

of the portfolio variance mitigates against traditional measures of reliability and

validity . . . The more authentic the personal variation [of portfoHo contents], the less

valid are traditional measures [for vahdity and reliabiHty]" (Ashehnan & Lenhoff,

1994, p. 75).

If there is no comparison to known standards, portfolios may lack

interpretabihty (Raybom, 1993). Raybom noted that while measurement

methodologies have not been developed that allow the products in a portfolio to be

quantified and aggregated, emerging technology could allow evaluators to do so.

28

In summary, portfoHos, as any good assessment method, should fit the

situation; no one type of portfolio is useful in every situation. Establishing the

purpose of the portfolio is the most important step in determining the format of the

portfolio (Seldin, 1993). The process of developing a portfoHo, using collaboration

and reflections, may be more important to the growth and improvement of an

individual than achieving a completed product. ReHabiHty and validhy of the portfolio

will be dependent on criteria estabhshed for the portfoHo's purpose and content.

Portfolios in Teacher Education

The ongoing discussion about improving schools and the increased focus on

teacher and smdent accountabUity have directed attention to the professional standards

of teachers and the programs that prepare teachers. The need to upgrade standards for

the teaching profession and to find appropriate ways to assess the performance of

teacher education programs has led to the development of the professional

development school: centers for teacher education that meet the demands of the

profession and of the pubHc for accountabihty (Darling-Hammond, 1990). As

curriculum and instmction change in professional development schools and in coUeges

of education, the means of assessment must change to better fit the resulting product.

A review of the Hterature on portfoHos in teacher education programs included

purposes and uses of portfoHos in teacher education. The content and organizational

methods of teaching portfoHos also were reviewed.

29

Purposes and Uses of Teaching Portfolios

The use of portfolios with preservice teachers in teacher education programs is

spreading; although, research efforts in the development and use of teaching portfolios

have only become focused in the last five years (Seldin, 1993). Touzel (1993)

maintained that "portfolio analysis" elicits superior and more vahd data from teacher

education majors than does standardized testing.

A teaching portfolio is a tool for communicating teaching accomplishments and

represents a complete picture of students' learning and growth (Grady, 1992; Arter,

1993; Wolf, LeMahieu, & Eresh, 1992). While conununicating the preservice

teacher's philosophy about teaching (Seldin, 1993), it also represents student-selected

work (Grady, 1992; Paulson, Paulson, & Meyer, 1991; Wolf, LeMahieu, & Eresh,

1992) and purposeful and meaningful class assignments (Maeroff, 1991).

Portfolios for preservice teachers serve the following purposes: help in seeking

employment (Grady, 1992; Feuer & Fuhon, 1992), model assessment strategies tiiat

students can use in their own classroom, document professional development, and

uiform and hnprove mstmction (Grady, 1992). Seldm (1993) asserted that tiie

teaching portfoHo may more accurately measure and judge a person's teaching

effectiveness than evaluations submitted by the teacher's students.

The Institute for Performance-Based Accountabihty (1994) suggested to tiie

Texas Conunission on Teaching Standards that each teacher education program m

Texas design hs own portfolio program and establish hs own criteria for analysis and

30

hiterpretation of portfolio data. A selected sample of portfolios from each institution

could be submitted to the appropriate state agency for evaluation.

Content of Teaching Portfolios

PortfoHo content wiU vary widely from institution to instimtion, department to

department, and teacher to teacher (Cramer, 1993). The portfolio content should be

based on the purpose, on course content, and on desired outcomes.

Preservice teaching portfolios typicaUy include the foUowing hems:

supervising teacher observations and evaluations, self-assessment questionnaires

(Grady, 1992), evidence of instmctional planning, evidences of experiences in working

with diverse populations (IPBA, 1994), video-taped lessons and other activities, a

collection of the work done by students in the preservice teacher's classes,

computerized data of any of the preceding items or evidence of teaching with

technology. Case study documents, either produced or critiqued by the preservice

teacher, a reflective journal, and the preservice teacher's observation records of classes

and students are additional items that might be included.

PortfoHo development is based on the principle of constmctivism. By

developing a portfoHo, the preservice teacher models behaviors expected of the learner

who constmcts meaning (Secretary's Commission on Achieving Necessary Skills

[SCANS], 1991) and becomes " . . . an active observer who coUects and documents

data and then evaluates and make sense of it" (Grady, 1992, p. 30). Ashehnan and

Lenhoff (1994) concluded that portfoHo development ". . . vaHdates the importance of

31

each student's role in self and shared reflection, goal setting and commitment to taking

personal responsibility for professional growth" (p. 66). The portfolio development

process allows for variance in the rate of student growth and promotes self-esteem

(Rich, 1994).

Portfolios requke tune to develop and review (Grady, 1992; Popham, 1993).

Their use requfres conscientious management of thne and materials. PortfoHos should

not be addhional paperwork for students but should be designed as a significant part

of the curriculum (Grady 1992). Angelo and Cross (1993) noted tiiat "Students may

spend so much tune selectmg or creatmg the components that they slight the task of

interpreting the contents of their portfolio" (p. 211).

Portfolios can provide a context in which students learn to view assessment as

an occasion for learning (Wolf, 1989). Shulman (1987) defmed the process of

reflection as ". . . reviewing, reconstmcting, reenacting and critically analyzing one's

own and the class's performance, and grounding explanations in evidence" (p. 31).

Sparks-Langer, Simmons, Pasch, Colton, and Starko (1991) noted that improved

reflection processes is a major goal in improving teacher education.

Organizing a Teaching Portfolio

The purpose of the teaching portfolio will influence decisions about the

organizational components of the completed product. A researcher-developed table

compares eight models of teaching portfolio organization (see Table 2.2). While the

number of organizational components in a teaching portfolio may vary, from three in

32

Seldm's model (1993) to five in models proposed by Green and Smyser (1996),

Scriven (1988), and Wolf (1991), all teaching portfolios will include products or

artifacts of the teaching experience. Knowledge of content, professional activhies,

service to others, and philosophy or reflections are common components.

The 1991 KilHon and Todham model (cited m Jensen & Shepston, 1992) presented a

unique focus on the perceived importance of reflection as a component of portfoHo

development.

In summary, teaching portfoHos are a dynamic, rather than static, means of

assessment, ever-changing, ongoing, and self-developing. "PortfoHos offer an exciting

vehicle to involve students in their learning and showcase theu achievements"

(Cramer, 1993, p. 74). Teaching portfoHos offer teacher education programs an

altemative assessment form that can be used in assessment of student achievement as

weU as in program assessment.

Research on Effectiveness of Portfolios

While the number of pubHcations on portfoHos is growing, the majority of the

writing has been anecdotal in nature: educators describing how they began using

portfoHos in their classrooms and the perceived resuhs of portfoHo development m

their classroom or school setting. Only a few programs have stmctured their portfolio

development process to measure the effectiveness of the product or the process.

The review of Hterature provided a varied group of research studies regarding

portfolio effectiveness. Topics included in this discussion are portfolio development

33

Table 2.2 Organizational Components of Teachmg Portfolios.

Author (s)

Seldin

Green & Smyser

Barton & Collins

Mathies & Uphoff

Scriven

Jensen & Shepston

KiUion & Todham (cited in Jensen & Shepston, 1992)

Ashehnan & Lenhoff

K Wolf

Date

1993

1996

1993

1992

1988

1992

1991

1994

1991

Organizational Components

Materials from Oneself Materials from Others Products of Good Teaching

Introduction Influences Instruction Individuahzation Integration

Artifacts Reproductions Attestations Productions

Professional ResponsibiUties Content Mastery Organization and Management Pedagogy

Knowledge of Subject Matter Instructional Skills Assessment Skills Professionalism Other Services to School

Professional Development Plan Artifacts of Teaching Reflections

The Reflective Portfolio: Reflection on Action Reflection in Action Reflection for Action

Writing Philosophy Simulations Field E;q)eriences

Background Information Unit Planning Examples of Instruction Studoit Assessment Professional and Community Exchange

34

as an assessment process, use of portfolios in the hiring process, and the impact of

portfolio development on teaching improvement. A portion of the research on

portfolio effectiveness concerned evaluation strategies for portfolios.

Portfolios as Assessment

In a study by faculty at Wayne State University, researchers sought to assess

the potential of the portfolio process in preparing teachers and in assessing the teacher

education program (Snyder, Elliott, Bhavnagri, & Boyer, 1993). An mitial pilot

program in 1991 was followed by a three-semester phase-in of portfolio development

as a requirement for completion of student teaching. During the Fall 1993 semester,

public school and university teachers and administrators were invited to serve as

reviewers for 236 portfolios developed by students during the semester. Reactions

from participants included the following: students asked for more and clearer

directions in developing the portfolios and more time than a single semester to develop

them; university faculty used the portfolios to improve their own instmction in

education classes; and 98% of the reviewers believed that the portfolios adequately

represented the preservice teachers' development. Both students and reviewers

expressed a high degree of enthusiasm for the portfolio development process and ched

professional growth and the exchange of ideas as two outcomes. The researchers

suggested that portfolios for job interviews may need to be developed differently and

that first-time reviewers may need more information on the purpose of the portfolio.

35

Hartle and DeHart (1993) reported on a constmctivist assessment plan with

elementary preservice teachers that included a professional portfolio as one component.

Feedback was favorable from supervising teachers, preservice teachers, and university

supervisors regarding the role of portfoHos in linking theory and practice. One initial

finding was that preservice teachers, generaUy inexperienced in self-assessment,

needed more stmcture than the preHminary guidelines for portfoHo development

provided. The researchers suggested using an active mentoring process to help

preservice teachers become effective self-evaluators.

Use of PortfoHos in Hiring

Smdies conducted by researchers associated with an Ohio teacher education

consortium examined the reactions to teaching portfolios of public school

administrators who hire faculty (Smolen & Newman, 1992; Newman, Smolen, &

Newman, 1993). The first study asked a group of administrators to respond to a series

of questions regarding their experiences with portfolios in the hiring process and their

wiOingness to use portfoHos for hiring purposes. The administrators showed a high

level of agreement on the foUowing essential portfoHo contents: resume, student

teaching evaluation, and recommendations by school personnel. Unit plans, audio or

video tapes, pictures of teaching, and evidence of community and volunteer service

were considered to be of Httle interest by administrators. Principals were more

interested in seeing concrete evidence of classroom skiUs; they rated unit and lesson

plans, classroom management plans, evidence of computer skiUs, and a personal

36

philosophy of education statement more useful than did other types of administrators

(vocational directors, superintendents, curriculum directors, and personnel directors).

The majority of administrators responded favorably to the idea of using

portfolios when reviewmg potential candidates, whh only one administrator mdicatmg

that he or she would not use the portfolio. The amount of tune that 80% of the

administrators were willmg to allot to a review of the portfolio was 3-15 minutes.

In the second study, reported by Newman et al. (1993), a smaUer group of

public school administrators responsible for hiring teachers were asked to review

teaching portfolios developed by preservice teachers. Administrators deemed the most

essential parts of the portfolio to be a grammatically-correct error-free presentation, a

college transcript, and a videotape of actual teaching.

Over two-thirds of the administrators in this study spent much longer viewing

the videotape than anticipated. While the authors did not discuss the disagreement

between the 1992 and 1993 studies concerning the importance of the videotape,

perhaps, the second group of administrators found the videotape compelling evidence

of proficient (or insufficient) teaching skills. All of the administrators indicated that

the portfoHos, including the videotape, provided sufficient information about the

candidate to make a hiring decision. The study examined mter-rater reliability in the

scoring of the portfolios. There was a high degree of correlation between portfolio

scores given by the administrators and scores assigned to the portfolios by university

faculty.

37

In a survey of administrators on the utility of 16 portfolio items, Bouas, Bush,

and Fero (1994) reported the following items in a teaching portfolio as the most

helpful: resume, certification, and letters of recommendation. The study found

administrators equally divided on the issue of videos as part of the portfolio. Some

administrators believed the video lacked validity in assessing a teacher's effectiveness;

others appreciated seeing the teacher m action. Many cited a lack of thne to watch

applicants' videos. The researchers, however, concluded that the inclusion of a video

could positively influence the hiring decision.

Portfolio Development and Teaching Improvement

Several large-scale assessment projects have included portfolios as a component

and have conducted studies to determine their effectiveness. The Teacher Assessment

Project (TAP) at Stanford University initiated classroom teaching portfolios as a means

of evaluating teachers for national certification (Wolf, 1991). Portfolios were used in

conjunction with other performance assessment strategies and opportunities for

dialogue between teachers and researchers. Artifacts of teaching and written

reflections on the meaning of the evidences were included in the portfolios.

The TAP study examined the possibilhy of scoring bias between the "glossy"

portfolio and one that was less polished in appearance. Reviewers in the study

consistently opted to reward substance rather than appearance. Two additional

fmdings of the study were that experienced teachers wanted a set of explich guidelines

38

and more direction in the development of the portfolio; a video tape was one of the

most important evidences of teaching.

A study conducted by Ohlhausen and Ford (1990) examined how the

experience of developing Hteracy portfolios m a graduate education class would unpact

the teaching and assessment practices m the students' own classrooms. Prior to the

class, approxunately two-thirds of the student group said they were usmg portfolios m

some way in then classrooms (primarily to collect student papers to share with

parents/guardians during conferences). FoUowmg the portfolio experience, ahnost all

the students indicated they would initiate or expand the use of portfolios whh theh

own students. A follow-up questionnaire revealed that while only 50% of the graduate

students were actually using portfolios in their own classrooms, 76% of those using

portfolios had increased the amount of student involvement in the development of the

portfolios.

A further finding of the study revealed that portfolios can increase the

portfolio-developer's insight into his or her own growth and development. While 85%

of the participants in the Ohlhausen and Ford study indicated that developing

portfoHos had increased their abiHties in self-assessment, a review of their portfolios

revealed that the graduate students' self-assessment abilities were limited. Though the

portfolio served as a vehicle to increase student thinking, the level of thinking of most

of the graduate students seldom directed plans for increased growth. In those

instances where students set goals, developed plans, and charted progress, " . . . the

portfolio process exceeded our expectations for providing evidence of growth and

39

thinking" (p. 3). The inability of the graduate students to effectively develop theh

own portfolios at higher levels of thinking and growth may have impacted the decline

in use of portfolios whh their own students. A fuller understanding of portfolio

development may have caused them to more clearly differentiate between portfolios

and folders of papers.

Geltner (1993) conducted an ethnographic study of students in an educational

administrator preparation program. The study found that systematic mentoring

sessions during the development of a professional portfolio for assessment purposes

increased metacognition and the ability to apply theory to practice. An additional

benefit of the mentoring sessions was the enhancement of leadership skills in

educational settings.

Evaluating Portfolios

An Alabama teacher education consortium sought to determine if scores on

altemative assessment measures such as performance portfolios, interviews, and essays

could be as effective as more traditional assessment measures, such as high school

grade point average (GPA), Scholastic Assessment Test (SAT) and American College

Testuig Assessment Test (ACT) scores, and selected course work, m determining

acceptance into the teacher education program (Nweke, 1991). Higher correlations

were found between portfoHo and interview scores and scores from sophomore-level

work, specificaUy an essay, than with scores from traditional measures secured prior to

the sophomore level. The author attributed the high correlation to the fact that

40

because portfolios measure growth over time, the sophomore essay would have

exhibited some common attributes with the portfolio. Nweke concluded that while

portfolios were judged useful in the assessment process, they should be used in

conjunction with more traditional measures for determining readiness to enter the

teacher education program.

A 1989 study by Jay Simmons (cited in Polin, 1991) showed no significant

difference between portfolio scores and scores on student tests among average and

above average students. Portfolio scores were significantly higher than standard scores

for those students who are most often victims of conventional educational testing

methods.

Collins (1993) reported that on the biology teacher portion of the Teacher

Assessment Project (BioTAP), the portfolio (only one component of the assessment

process) proved to be a feasible means of assessing the performance of teaching.

Ratings of the portfolios and the other assessment activities were consistent across the

raters; variations in the ratings were consistent whh the raters' backgrounds. The

teacher participants valued the portfolio, notuig h ". . . had the highest fidelhy to theh

practice and [the follow-up exercise with the portfolio] gave them the opportunity to

taUc about what tiiey did" (p. 1117).

The Vermont portfolio project, one of the pioneermg efforts m large-scale

portfolio assessment, was a statewide samplmg of student portfolios to assess the level

of math and Hteracy mstmction m Vermont schools (Polhi, 1991). Early efforts to

score the portfolios produced disappointmg resuhs, but the enthusiasm for the process

41

remained high. Vermont continued to use the portfolios statewide and developed a

team of trained reviewers to score the portfolios.

In summary, the teaching portfolio is a tangible record of experience that

shows progress from the beghuiing of the preservice program through student teachmg

and beyond and is effective m assessmg student progress (IPBA, 1994; Cramer, 1993;

Reilly et al., 1993). Obtahimg baselme data about student performance is critical to

any assessment model. Portfolios offer one means of " . . . capturing baseline data

and adding data over tune which can clearly show gain or change" (French, 1992, p.

260). The teaching portfolio can demonstrate the extent to which preservice students

meet the leamer-centered proficiencies established for Texas educators.

Summary

Assessment reform and altemative assessment strategies are topics of wide

interest in education. The use of portfolios as a form of altemative assessment has

generated research studies from kindergarten through university. Teacher education

programs are beginning to use portfolios as an authentic strategy for assessing teaching

proficiencies. A variety of organizational components are suggested for teaching

portfolios.

This chapter attempted to provide background on issues of interest to the

researcher in developing a portfolio prototype Issues relevant to this study included

assessment and altemative assessment strategies. Research studies on portfolios as an

altemative assessment tool, use of portfolios in teacher education, and portfolio

42

effectiveness were reviewed. Summary tables on models of altemative assessment and

organizational components of teaching portfolios, developed by the researcher from

this review of the literature, are mcluded.

While there are unresolved obstacles in the portfolio development process,

users of portfolios remain enthusiastic about their potential. Teaching portfolios can

be useful in assessing both the quality of a teacher education program and the extent

to which preservice teachers can demonstrate mastery of teaching proficiencies.

43

CHAPTER m

METHODOLOGY





The evidences of the proficiencies were selected by consensus of three stakeholder


Texas pubHc school personnel administrators who are responsible for hiring home




A discussion of the research design is provided in this chapter, including an

explanation of the Delphi process and its modification for this particular study.

Selection of subjects, instrument design, and procedures for the two-round Delphi

process are given.

Research Design

This study used an exploratory research and development design. The research

and development procedures are divided into (1) collection of data using a two-round

Delphi process and an accompanying questioimaire, discussed in this chapter, and

44

(2) development of a portfolio prototype based on data collected during the Delphi

process, discussed in Chapter V.

Selection of Subjects

The populations for this study were home economics teacher educators

employed in teacher education programs accredhed by the Texas State Board of

Higher Education, Texas public school personnel administrators who are responsible

for hiring home economics teachers, and home economics preservice teachers engaged

in student teaching during spring 1995. Each of the participant groups are

stakeholders in providing quality education for Texas pubhc school smdents.

The entire population of home economics teacher educators in Texas (n = 26)

comprised the sample. The list of home economics teacher educators in Texas was

obtained from Texas Education Agency, Home Economics Program Division, and

represented 13 universities in Texas (see Appendix A). Because all home economics

teacher educators in Texas were surveyed, the resulting data should be generalizable to

all home economics education programs in the state of Texas.

The population of home economics preservice teachers consisted of home

economics preservice teachers engaged in student teaching in Texas during the spring

1995 semester. The list of preservice teachers {n = 40) was provided by nine home

economics teacher education programs in Texas. Two of the home economics

teaching units had no home economics preservice teachers during spring 1995. One

university's home economics teacher educators were teaching in areas of specialization

45

m home economics rather than in home economics education. Confidentiality issues

prohibited one home economics education unit from releasing a Hst of its preservice

teachers.

Texas pubHc school personnel administrators who are responsible for hiring

home economics teachers in a school district comprised the third population for the

smdy. Studies by Smolen and Newman (1992), Newman, Smolen, and Newman

(1993), and Bouas, Bush, and Fero (1994) found tiiat administrators considered

portfolios useful in making hiring decisions. Administrators' ideas and concems

should be considered when preparing professional portfoHos.

A random sample of 30 Texas pubHc school districts from among aU Texas

pubHc school districts was selected. The Hst of school districts was obtained from the

Texas School Directory (1994). Hinkle, Wiersma, and Jurs (1988) noted that

systematic sampling from a Hst (using every /rth member of the population) provides a

convenient way to sample throughout the population by spacing the selections over the

entire population Hst. Gay (1992), however, pointed out that systematic sampling of a

nonrandom Hst (alphabetical, for example) may exclude certain groups and include

greater numbers of large groups with similar names. Gay contended that either the

process or the Hst needs to be randomized.

Because the school districts were listed alphabetically, a random sample of the

population was selected using a table of random numbers from Gay (1992). The

school districts were numbered from 0-1057. After selecting an initial number from

46

the list of random numbers, school districts were randomly selected from the list of all

school districts in Texas.

As each school district was randomly selected and in order to assure that the

school districts were representative of all parts of the state, the researcher noted the

Education Service District in which each school district was located. Education

Service Districts were authorized in 1965 by the Texas Legislature to provide regional

media centers m aU geographic areas of the state. The 20 Education Service Centers,

centraUy located within each Education Service District, develop federaUy-funded

educational projects and provide services to member school districts, including staff

development, accreditation, data management, textbook review procedures, and

curriculum development and implementation (Funkhouser, 1994).

Once two school districts had been selected from any one Education Service

District, no additional school districts were selected from that district. Selection

continued until a Hst of 30 school districts and alternate school districts, representing

aU Education Service Districts in Texas, was obtained. Telephone caUs were made to

each of the 30 school districts to secure the name of the personnel administrator

responsible for hiring home economics faculty for the school district.

Two of the original 30 selected school districts did not have home economics

programs; these were replaced with the first two alternate school districts representing

the same Education Service Districts. The identified personnel administrators (n = 30)

from the school districts in the sample were surveyed using the Delphi process.

47

The Delphi Process

The Delphi process was selected as the method for seeking consensus on

evidences that authentically assess mastery of the five teaching proficiencies. The

researcher has included a review of the literature on the Delphi process as a basis for

the modified Delphi process used in this study.

The Delphi process is a group surveying procedure consisting of sequential

probing phases that solicit information on a specific topic. Used for forecasting and

developing ideas and proposals in business, education, and medicine (Eason, 1992),

the Delphi process (with numerous variations) is a scientific inquiry method used since

ancient Greece. In ancient Greece, the Oracle at Delphi would consider proposals

from a group of anonymous participants, select the proposal that would best fit the

given objective, and resubmit the chosen proposal for further consideration and

revision (Broadbent, 1989).

Sackman (1975) summarized the Delphi process as " . . . an attempt to eHch

expert opinion m a systematic manner for useful resuhs" (p. xi). Todd and Reece

(1989) described the process as a ". . . metiiod of arrivmg at a consensus from a group

of experts witiiout bringmg the respondents together" (p. 3). Lmstone and Turoff

(1975) referred to the Delphi process as " . . . a group conununication process . . .

effective hi allowmg a group of mdividuals, as a whole, to deal with a complex

problem" (p. 4). The Delphi process is appropriate for this study because three

distmct groups of experts were bemg asked to reach consensus on tiie muhi-faceted

48

problem of selecting evidences to authentically assess mastery of the five teaching

proficiencies.

Each round of questioning or probmg m the Delphi process is followed by a

smnmarization of the responses and a controlled feedback to tiie panel, often

resubmission to tiie panel of tiiose hems that have the highest consensus (Linstone &

Turoff, 1975; North & Pyke, 1969; Van de Ven & Delbecq, 1974). The process

contmues until sufficient information is gained to test the solution (Sackman, 1975) or

to develop an operational plan (Eason, 1992; Lmstone & Turoff, 1975).

North and Pyke (1969) noted that the Delphi process reduces negative aspects

of group interaction. The participants of the Delphi process are anonymous to each

other; therefore, any ideas or revisions will be considered on merit alone and will not

be evaluated in light of the personality or the status of the person proposing the idea

or revision. This keeps bias to a minimum and " . . . helps ensure the validity of the

results" (Todd & Reece, 1989, p. 4).

Another advantage of using the Delphi process is that issues can be resolved

and consensus reached without time and/or money being spent in travel. Because the

group of experts do not meet face-to-face during the probing process, the place of

residence of each participant is not an hnportant consideration (Todd & Reece, 1989).

The Delphi process is appropriate to use when conducting a statewide survey such as

this study.

The Delphi process is particularly useful hi an exploratory research study

because it can accommodate a variety of opinions, utilize diverse types of questions.

49

and allow flexibility in the direction that the research takes (Eason, 1992). A usual

first round in the Delphi process allows participants to provide input on the criteria to

be used in the problem-solving process, ff a list of criteria is already established, this

can be presented to the participants for ranking for importance or for further revision,

ff there is not a clearly defmed topic, open-ended questions presented to the

participants in an interview or written survey can allow for a variety of responses.

The subsequent round(s) of questioning, based on summaries of responses from the

previous round, may seek to gain additional input on the list of responses, ask

participants to rank-order the responses, or develop addhional themes or suggest a plan

based on the current Hst of responses.

This particular study omitted the usual fust round of input and began with a

list of evidences gathered from a review of literature and discussion with home

economics teacher educators at two Texas state universities. Responses to the round-

one questionnaue shaped the round-two questionnaire, and responses to rounds one

and two dfrected the development of the portfolio prototype.

Sackman (1975) highHghted several problems witii the Delphi process:

1. The group of chosen or self-selectmg participants may not always

represent vaHd "expert" opinion.

2. While the process may allow for a variety of mput, the consensus

process attempts to pare away any deviant opmions or exploratory ideas.

The consensus reached by the group may not represent the best thoughts

of the group and may resuh m ". . . compounded ambiguhy" (p. 70).

50

3. The anonymity of the respondents creates questions about

accountabihty.

In an effort to overcome the expert opinion challenge, the enthe population of

home economics teacher educators in Texas was selected. The entire population of

home economics student teachers in Texas for Spring 1995 also was selected;

however, only 40 names and addresses were provided by nine home economics teacher

education units in Texas. The random sample of Texas public school personnel

administrators responsible for hiring home economics teachers represents both large

and small school districts from ah areas of the state.

To resolve the chaUenge of consensus leading to ambiguous results, only two

rounds of the Delphi process were used. Todd and Reece (1989) found that m a third

round of probing for consensus, the majority of the panel ignored the new items added

to the questionnaire. The decline in contributing responses was attributed to a lack of

time and/or a decrease in mterest. A third round, therefore, was not used in this study

in the interest of time and efficiency.

The decision to use the Delphi process in the present study was based on hs

mherent characteristics: interactivhy among group members desphe then place of

residence, lunited influence from personalhy or status of tiie group members, and

flexibiHty hi the type of data coUected (either stmctured or open-ended questions)

(North & Pyke, 1969; Todd & Reece, 1989; Eason, 1992). While tiiis study sought

creative ideas from the respondents for authentic assessment of the five leamer-

centered teaching proficiencies, a consensus on the evidences that authentically show

51

mastery of the proficiencies was deemed important in developing a portfolio prototype

that could be useful in assessing mastery of the proficiencies.

Instrument Design

In order to gather data on evidences that authentically assess mastery of the

five leamer-centered teachmg proficiencies (see Table 3.1), a checklist was developed

for use in a modified two-round Delphi process. A researcher-designed questionnaue

was included in round one to gamer background and demographic information from

participants. The round-two checklist was designed following the retum of round-one

responses.

Round-One Instrument. The round-one, researcher-designed instrument was

pre-tested with faculty members at two Texas state universities. Revisions to the

instrument were made prior to the instrument being sent to the participants.

The round-one instrument (see Appendix B) consisted of three sections:

introduction, checklist, and background and demographic mformation. The fust

section provided a brief introduction to the study. The second section, used for the

Delphi process, consisted of a checklist for each of the five leamer-centered teaching

proficiencies. Checklists for each proficiency mcluded 30 evidences that might be

used to assess mastery of the leamer-centered proficiencies. The evidences, listed in

Figure 3.1, mcluded the most conunonly suggested assessment hems for teaching

52

Table 3.1 Leamer-centered Teaching Proficiencies Adopted in Texas. ^

Proficiency

Leamer-centered Knowledge

Leamer-centered Instruction

Equity in Excellence for All Leamers

Leamer-centered Communication

Leamer-centered Professional Development

Proficiency Statement

The teacher possesses and draws on a rich knowledge base of content, pedagogy, and technology to provide relevant and meaningful learning experiences for all students.

To create a leamer-centered conmiunity, the teacher collaboratively identifies needs and plans, implements, and assesses instmction using technology and other resources.

The teacher responds appropriately to diverse groups of leamers.

While acting as an advocate for all students and the school, the teacher demonstrates effective professional and interpersonal communication skills.

The teacher, as a reflective practitioner dedicated to all students' success, demonstrates a commitment to learn, to improve the profession, and to maintain professional ethics and personal integrity.

^ Learner-Centered Schools for Texas: A Vision of Texas Educators, 1994.

portfoHos (Bouas, Bush, & Fero, 1994; Seldm, 1993; Zolhnan & Jones, 1994) as well

as evidences of importance to home economics programs suggested by home

economics teacher educators.

Delphi participants were asked to mark all evidences that best assess each of

the five leamer-centered teaching proficiencies. An other category allowed mput from

respondents on addhional evidences of mastery of each teaching proficiency.

This research modified tiie procedures used by Dodge and Clark (1977) and

Cyphert and Gant (1970) as a basis for pre-determmmg the levels of consensus for

53

responses to the Delphi process. Both of the previously-mentioned studies provided

participants the interquartile range (the middle 50 percent) of responses as a basis for

subsequent decision making.

Concem about the abiHty of three diverse groups to reach consensus on the

evidences at 75% prompted the researcher to use a range of 20%-70% agreement as

the range of hems of evidence to resubmit for further consideration by participants.

Thus, items of evidence achieving 70% agreement in round one were determined to

have achieved consensus. Items receiving less than 20% agreement were not

resubmitted in the second round of the Delphi process. Items of evidence achieving

between 20% and 70% agreement in round one were resubmitted to round-one

respondents to see if consensus could be attained.

Section three of the round-one instrument was a researcher-designed

questionnaire that eHcited specific responses from each participant group on a variety

of background and demographic items that related to the objectives of the study (see

Appendix B). Stmctured (closed-form) questions that provide a set of responses were

used to facilitate responding and scoring (Gay, 1992). Common questions for each

participant group sought responses on educational background and teaching or

administrative experience.

54

Achievement Awards/Certificates

Class Projects Related to Teaching Field

Classroom Procedures/Discipline Plan

Creative Activity

Evidence of Community Service

Evidence of Computer Skills

Description of Teaching SUategy Used

Documentation of Field Experiences Prior to Student Teaching

EnUepreneurship Plan

Evaluation Devices (Teacher Designed)

ExCET Scores

FHA/HERO or 4-H Activities Log

Instmctional Materials (Teacher Designed)

Leadership in Professional Associations

Letters of Recommendation

Membership m Professional Associations

Philosophy/Goals Statement

Photographs of Teaching Activities

Professional Development Activities (i.e., Meetings Attended)

Pubhc Relations Project

Pupils' Evaluations

Reflective Journal of Student Teacher

Research Project

Resume

Samples of Pupils' Work

Self-assessment ChecklisVEssay

Student Teachmg Evaluations (from Supervismg Teacher, University Supervisor, etc.)

Transcript

Unit/Lesson Plans

Video/Audi otape of Lesson

Figure 3.1 Evidences Proposed to Assess Mastery of the Teaching Proficiencies.

55

Home economics teacher educators were asked about the current level of

portfolio use in subject-matter courses or in supervision of student teaching. Texas

public school personnel administrators who hire home econonucs facuhy were asked

to respond to questions regarding theu experiences whh portfolios as a means of

evaluating candidates for hiring. Preservice home economics teachers were surveyed

about tiieir use of portfolios for improvement of teaching and for employment.

A cover letter on univershy letterhead (see Appendix A), co-signed by the

researcher and the dissertation committee chair, accompanied the round-one

instrument. The letter emphasized the hnportance of the study and hs relevance to

current and future classroom practices and asked for a commitment to complete two

rounds of the Delphi process.

Both round-one and round-two instruments included an addressed and stamped

9 X 12 envelope for convenient retum. Respondents were provided the opportunity to

remain anonymous if desired. Questionnaires were coded for follow-up purposes only.

Home economics teacher educators and preservice teachers were asked to

submit examples of assignments and/or student work that would be appropriate to

include in a portfolio assessing the leamer-centered proficiencies. A release form (see

Appendix B) was included with both round-one and roimd-two instruments for those

who chose to submit examples for use in the proposed portfolio prototype.

Appropriate citation of any submissions included in the portfolio prototype is given in

the dissertation acknowledgements (see p. u).

56

Round-one instruments were mailed to home economics preservice teachers,

home economics teacher educators, and personnel administrators in September 1995.

In cases where the round-one instrument was not returned within two weeks, a follow-

up letter (see Appendix A) and copy of the instrument were mailed, ff, at the end of

the next two weeks, a reply was not received, a telephone contact was attempted to

secure a response. Responses from round one are shown m Table 3.2.

Table 3.2 Summary of Responses to Round One of the Delphi Process.

Participant Groups

Preservice Teachers (N=40)

Teacher Educators (N=26)

Personnel Administrators (N=30)

TOTALS (N=96)

N

20

19

21

60

%

50

73

70

63

Round-Two Instrument. The round-two checklist for the Delphi process (see

Appendix C) was developed after responses from round one were received and

analyzed. Evidences of each of the teaching proficiencies in round one that received

at least 70% consensus were accepted as evidences that could authentically assess

mastery of that particular teaching proficiency and were not included in the round-two

checkHst. Those evidences receivmg less than 20% consensus in round one were

omitted from round two.

The remaining evidences under each teaching proficiency were resubmitted to

round-two participants in an attempt to gain consensus on additional items. Addhional

57

evidences suggested by round-one respondents were added to the round-two

questionnaire under the appropriate teaching proficiency and were marked with an

asterisk to denote a new item.

Checklists for the round-two Delphi process were mailed in May 1996 to

members of the participant groups responding to round one. Round-two participants

included 19 home economics teacher educators, 20 preservice home economics

teachers, and 21 Texas public school personnel administrators. Telephone contact was

attempted with non-respondents to ensure a higher retum rate. Table 3.3 summarizes

responses to round two.

Table 3.3 Summary of Responses to Round Two of the Delphi Process.

Participant Groups

Preservice Teachers (N=20)

Teacher Educators (N=19)

Personnel Administrators (N=21)

TOTALS (N=60)

N

12

17

9

38

%

60

89

43

63

No evidences received 70% consensus in round two of the Delphi process.

The proficiency Learner-centered Communication had no evidences reaching 70%

consensus in either round one or round two.

To be able to have examples of evidence for each of the five leamer-centered

teaching proficiencies represented in the portfolio prototype, it was necessary to retam

at least one evidence under each proficiency. In the literature there is no standard of

58

consensus for the Delphi process with some studies accepting consensus as low as

50%. For the proficiency Leamer-centered Communication, Evidence of Parent

Contact in Student Teaching achieved highest agreement in round two, receiving

sHghtly over 60% agreement. The researcher decided to accept evidences for all

proficiencies that received 60% or higher consensus in round two.

A response card (see Appendix A) was included with the round-two checkHst

asking respondents if they were interested in participating in the portfoHo prototype

evaluation process. A total of 27 cards were returned with round-two Delphi

responses. Nineteen respondents indicated an interest in serving on the panel to

review the portfoHo prototype. The procedures for developing the portfoHo prototype

are described in Chapter V.

Data Analysis

Data secured through the survey instruments were analyzed foUowing each

round of the Delphi process. Results are reported in Chapters VI and V through

descriptive statistics (frequencies, percentages, and means). According to Borg and

GaU (1989), the mean is the most stable method for reportmg a measure of centtal

tendency, although h is more likely to be affected by extreme scores than are the

median (middle score) or mode (most frequently occurring score). Frequency

distributions and percentages are useful m aUowhig the reader to easily understand

categorical data and dichotonues.

59

Sununarv

Following a review of Hterature on assessment and teaching portfolio

development, a modified two-round Delphi process ehched data from home economics

teacher educators, Texas public school personnel administrators responsible for hiring

home economics teachers, and home economics preservice teachers concerning

evidences that authentically show mastery of the five leamer-centered teaching

proficiencies adopted by the Texas State Board of Education. The data from the

round-one questionnaire and the two checkHsts used in the Delphi process are

discussed in Chapter IV. The portfolio prototype development and usability data are

described in Chapter V.

60

CHAPTER IV

FINDINGS AND DISCUSSION

Based on the review of literature and the statement of the problem for this

study, four research questions were proposed. Findings for three of the research

questions are reported in this chapter: characteristics of the participant groups,

portfolio use in assessment or hiring, and evidences that show mastery of the five

leamer-centered teaching proficiencies outlined in Learner-Centered Schools for Texas

(1994). The remaining research question is discussed and the portfolio prototype

development is described in Chapter V.

Findings of Research Questions One, Two, and Three

The purposes of this study were to identify evidences that authentically assess

mastery of the five leamer-centered teaching proficiencies adopted for Texas educators

and to develop a portfolio prototype, based on the identified evidences, for assessment

of home economics preservice teachers. The evidences of the proficiencies were

selected by consensus of three stakeholder groups: home econonucs preservice

teachers, home economics teacher educators, and Texas public school personnel

administrators who are responsible for hiring home economics teachers. The five

proficiencies outlined in Learner-Centered Schools for Texas (1994) are performance-

based standards for preparing educators and serve as an accountability system for

educator preparation programs in Texas.

61

The participants in this study were drawn from three distinct groups in Texas:

1. home economics teacher educators currently employed in teacher

education programs accredited by the Texas State Board of Higher

Education,

2. Texas public school personnel administrators who are responsible for

hiring home economics facuhy, and

3. home economics preservice teachers engaged in student teaching during

spring 1995.

The entire population of home economics teacher educators in Texas {n=26)

comprised the sample. A Hst of 40 preservice home economics teachers engaged in

smdent teaching in Texas during the spring 1995 semester was provided by home

economics teacher education programs in Texas. Thirty Texas public school personnel

administrators responsible for hiring home economics teachers were selected by

randomly sampling all Texas public school districts (N=1058).

The content of the portfoHo prototype was based on those evidences suggested

through a two-round Delphi process of the three stakeholder groups. Round-one

instruments were mailed in September 1995; round-two checklists were mailed in

May 1996. Summary responses from rounds one and two of the Delphi process are

shown in Table 4.1.

62

Table 4.1 Summary Responses for a Two-Round Delphi Process.

Participant Groups

Preservice teachers

{n = 40)

Teacher educators

{n = 26)

Personnel administrators

in = 30)

TOTALS (N = 96)

Round 1

N

20

19

21

60

%

50

73

70

63

Round 2

N

12

17

9

38

%

60

89

43

63

Total Responses from

Both Rounds

%

30

65

30

40

Research Ouestion 1

Research question 1: What are selected characteristics of the study's

participants: home economics teacher educators, home economics preservice teachers,

and Texas public school personnel administrators responsible for hiring home

economics teachers?

The round-one instrument included a questionnaue that eliched background

information from each of the three participant groups. Information obtamed in the

questioiuiake enabled the researcher to describe the participants from which the data

were coUected. A summary of the background data obtained from each of the three

participant groups follows.

63

Home Economics Preservice Teachers

Fifteen of the 20 home economics preservice teachers responding to round one

were enrolled in undergraduate home economics teacher education programs. The

remaining five respondents were emoUed as post-baccalaureate students.

Preservice teacher respondents were queried about theu employment goals.

Only one respondent, a post-baccalaureate student, was employed as a Texas home

economics teacher for Fall 1995. Seventy-five percent {n = 15) of the preservice

teacher respondents were seeking home economics teaching jobs in Texas, while one

respondent was seeking a teaching job outside of Texas. Two respondents (10%) were

seeking positions with the Texas Agriculture Extension Service, and two respondents

were not seeking employment. Two respondents were seeking jobs in other

employment areas rather than teaching; one was seeking employment in fashion

merchandising, and another was seeking employment as a curriculum specialist. Table

4.2 summarizes the profile of home economics preservice teacher respondents to this

study.

Home Economics Teacher Educators

Nineteen home economics teacher educators (73%) responded to the round-one

instrument. Information on teaching experience at college and public school levels

was requested. Eighty-five percent {n = 16) of home economics teacher educators

responding had taught at the coUege level for more than 10 years with 32% of the

64

Table 4.2 Proftie of Home Economics Preservice Teacher Respondents.

Profile Characteristics

Teacher Certification Program Enrollment Undergraduate Teacher Education P*rogram Post-baccalaureate Program

Employment Goals Currently Not Seeking Employment Employed for Fall 1995 as a

Home Economics Teacher Home Economics Teaching Job in Texas Home Economics Teaching Job Another State Home Economics Extension Position Another Employment Area

n ' (n = 20)

15 5

2

1 15 1 2 2

%

75 25

10

5 75 5 10 10

' Nimibers and percentages do not total 100 because respondents could respond to more than one item or because of rounding.

total {n = 6) having taught more than 21 years. Fifty-three percent {n = 10) of the

home economics teacher educators had taught m pubHc schools five years or less;

whUe 26% had 6-10 years of public school experience. Only three (16%) of the home

economics teacher educators had more than 10 years pubHc school experience; one

teacher educator had never taught in pubHc schools.

One general information question asked home economics teacher educators to

indicate the number of smdent teachers they had duectly supervised in field-based

experiences (i.e., smdent teaching or internships) shice August 1993. The question

was framed in this manner to accommodate those teacher education programs that

have home econonucs student teachers on ahematmg semesters. The time period,

from August 1993 through May 1995, mcluded four semesters.

65

Thirty-two percent of respondents indicated that they had no direct supervision

of student teachers during the four-semester time frame. Six teacher educators (32%)

indicated they had supervised 1-4 student teachers during the period; one teacher

educator had supervised 9-12; four (21%) had supervised 13-16; and two (11%) had

supervised 29 or more. Table 4.3 provides a profile of home economics teacher

educator respondents.

Table 4.3 Profile of Home Economics Teacher Educator Respondents, {n = 19)

Profile Questions

Number of years taught at the college level 0 - 5 years 6 - 1 0 years

1 1 - 1 5 years 16 - 20 years 21+ years

Number of years of pubUc school teaching 0 years 1 - 5 years 6 - 1 0 years

1 1 - 1 5 years 16 - 20 years 21+ years

Nxmiber of student teachers directly supervised in field-based experiences between August 31, 1993, and May 1995

0 1 - 4 5 - 8 9 - 12

1 3 - 16 17 - 2 0 21 - 2 4 25 -28 29 or more

in = 19)

2 1 6 4 6

1 10 5 2 1 -

6 6 -

1 4 -

-

-

2

%

11 5 32 21 32

5 53 26 11 5 -

32 32 -

5 21 -

-

-

11

' Numbers and percentages do not total 100 because respondents could respond to more than one item or because of rounding.

66

Personnel Administrators

In order to acquire a more representative sample of school personnel

administrators, an attempt was made to have administrators from each of the 20

Education Service Districts in Texas. Eighty percent (N=16) of the 20 Education

Service Districts established to provide educational support to school districts in Texas

were represented by personnel administrators responding to round one. The districts

shaded on the map in Figure 4.1 were represented in this study.

School districts in Texas are categorized, according to size of emoUments,

from 1A to 5A (University Scholastic League, 1996). Personnel administrators

responding to round one represented aU school sizes fairly evenly except for 3A and

4A schools. Table 4.4 shows the distribution of personnel administrator respondents

according to school-size categories in Texas.

Of the 21 personnel administrators who responded to round one, 76% were

male and 24% were female. Nine of the respondents (43%) were high school

principals; six (29%), superintendents; and six (29%), personnel duectors. Ten percent

of the personnel administrators had completed a doctorate, while the remainder

{n = 19) held a master's degree. One respondent had completed two master's degree

programs.

67

Regional Education Service Center

Region Headquarters Region Headquarters

1 2 3 4 S 6 7 8 9 0

Edinburg Corpus Christ) Victoria Houston Beaumont Huntsville Kllgore Mount Pieasant Wichita Fails Richardson

t1 ^2 13 14 15 16 17 18 19 20

Fort Worth Waco Austin Abilene San Angelo Amarllio Lubbock Midland Et Paso San Anionic

Figure 4.1 Education Service Districts hi Texas.

68

Table 4.4 Representation of Personnel Administrator Respondents According to Texas School-Size Categories, {n = 21)

Size Categories

lA School (149 students or less) 2A School (150-304 sttidents) 3A School (305-699 students) 4A School (700-1649 students) 5A School (1650 students or more)

n (n = 21)

4 4

7 1 5

%

19 19 33 5

24

Personnel administrators were asked about their pubHc school teaching and

admirustration experience. Forty-three percent of the respondents had taught for 21

years or more, while 66% (n = 14) had between 15 and 20 years of teaching

experience. Six personnel administrators (29%) had 21 or more years of

administrative experience.

Sixty-two percent {n = 13) of the personnel administrators responding to round

one had interviewed 15 or fewer teacher candidates in the previous school year. Five

respondents had mterviewed between 31-50 candidates, and three respondents had

mterviewed 76 or more candidates. Table 4.5 provides a profile of the personnel

administrator respondents.

69

Table 4.5 Profile of Personnel Administrator Respondents, {n = 21)

Profile Characteristics

Current Position High School Principal Superintendent Persormel Director

Gender Female Male

Highest Level of Education Bachelor's Degree Master's Degree Doctorate

Number of Years of PubUc School Teaching Experience 0 years 1 - 5 years 6 - 1 0 years

11 - 15 years 16 - 20 years 21+ years

Number of Years of PubUc School Administrative Experience 1 - 5 years 6 - 1 0 years

11 - 15 years 16 - 20 years 21+ years

Number of Teacher Candidates Interviewed in Past Year 0 1 - 15

16- 30 31 - 5 0 51 - 7 5 76 or more

n ' (n = 21)

9 6 6

5 16

19 2

1 3 3 5 9

5 5 2 3 6

13

5

3

%

43 29 29

24 76

90 10

5 14 14 24 43

24 24 10 14 29

62

24

14

Numbers and percentages may not equal 100 because of nonresponse and rounding.

70

Research Question 2

Research question 2: How are portfolios currently being used by home

economics teacher educators, home economics preservice teachers, and personnel

administrators responsible for hiring home economics teachers in Texas?

Participants in each stakeholder group were asked to rank a set of statements

concerning portfolio development or use. Preservice teachers ranked reasons for

developing a portfoUo, teacher educators ranked importance of portfoHos to a teacher

education program, and personnel administrators ranked portfolio items of value in the

hiring process.


Sixty-five percent {n = 13) of the home economics preservice teachers who

responded to the round-one questionnaire had developed a professional portfolio. Of

those respondents who had developed a professional portfolio, 10 had developed a

portfolio as a requirement in a course, while nine had developed a portfolio during

student teaching. Six respondents, therefore, had developed more than one portfolio.

Professional portfoHos had been used in an mterview for a teaching poshion by

23% of the preservice teachers who had developed portfoHos. One respondent had

used a professional portfoHo m mterviews for both teaching and non-teaching

positions. The non-teaching poshion was in the human services field. Table 4.6

summarizes home economics preservice teacher respondents' experiences with

portfoHos.

71

Table 4.6 Home Economics Preservice Teachers' Experiences With Portfolios.

Portfolio Experiences

Never Developed a Professional PortfoUo

Developed a PortfoUo as a Course Requirement

Developed a PortfoUo During Smdent Teaching

Used Portfolio in Interview for Teaching Position

Used Portfolio in Interview for Non-teaching Position

n ' in = 20)

7

10

9

3

1

%

35

50

45

15

5

Percentages may not total 100 because respondents were asked to select all answers that applied to their situation.

Home economics preservice teachers were asked to rank five reasons for

developing a portfolio, using a numeric ratmg scale from 1 {most important) to 5

{least important). Table 4.7 reveals the responses from both those who had developed

professional portfolios {n = 13) and those who had not developed portfolios {n = 7).

To market myself was rated most important by all preservice teacher respondents.

Table 4.7 Preservice Home Economics Teachers' Ranking of Reasons for Developing a Professional Portfolio, {n = 20)

Reasons for Developing a Professional PortfoUo

scale: 1 (most important) - 5 (least important)

To market myself

To demonstrate proficiency in teaching

To demonstrate knowledge about a subject

To demonstrate personal development

To draw closure on my teaching preparation

Mean of AU

Respondents

1.79

2.47

3.47

4.05

4.42

Had Developed a

PortfoUo

Mean

1.67

2.92

3.58

2.50

4.33

Never Developed a

PortfoUo

Mean

2.00

1.71

3.29

3.43

4.57

72


Three questions asked of teacher educators dealt with the use, the hnportance,

and the content of portfolios. Fifty-three percent of the teacher educators indicated

they did not require portfoHos in theh courses. Of the nine (47%) teacher educators

who used portfolios, seven requued portfolios in acadenuc classes only and one

required a portfolio in student teaching only. One home economics teacher educator

required students to develop portfolios both m student teaching and in academic

classes.

Teacher educator respondents were asked to rank the importance of three

purposes of portfolios to their teacher education program. A three-part response mode

was utilized with 1 as most important and 3 as least important. The responses are

shown in Table 4.8.

Table 4.8 Ranking by Teacher Educators as to Purposes of Portfolios in Teacher Education Programs, {n = 19)

Purposes of Portfolios scale: 1 (most important) - 3 (least important)

Markets competencies and skills

Shows proficiency in teaching

Shows mastery of content

Mean

1.43

2.00

2.57


Fifty-two percent {n = 11) of the personnel administrators had reviewed

professional portfolios submitted by teacher candidates during the interview process.

73

Of those 11 personnel administrators, six were personnel duectors, four were

superintendents, and one was a high school principal.

Two questions provided information about how and when portfolios were

reviewed by personnel administrators. Ten of the 11 administrators (90%) who had

reviewed professional portfolios said they reviewed the portfolios before the interview;

27% reviewed the portfolio during the interview, and 36% reviewed the portfolio

following the interview. Seventy-three percent of the administrators reviewed the

portfolio alone; 55% shared the review process with coUeagues, including campus-

level interview committees. Only 36% of the personnel administrators reviewed the

portfolio with the interviewee.

The amount of time spent reviewing each prospective teacher's portfolio varied.

Seventy-three percent of the respondents indicated that they spent fewer than 15

minutes on each portfolio, while 36% representing three superintendents and one

personnel director, indicated that they spent more than 20 minutes reviewing each

portfolio. Of the nine personnel administrators who had never had a teacher candidate

submit a professional portfolio in the interview process, more than half (n = 5)

indicated they were willing to spend from 6 to 15 minutes reviewmg a portfolio; four

were wiUing to spend more than 15 mmutes. Table 4.9 provides a summary of the

personnel administrators' experiences with professional portfolios.

74

Table 4.9 Experiences of Personnel Administrators with Professional Portfolios, (n = 21)

Experiences with Portfolios

Number of Teacher Candidates Interviewed in Past School Year (n=19)

0 1 15

16-30 31 -50 51 -75 76 or more

Number of Teacher Candidates Submitting Portfolios in Past School Year (N=19)

none 1-5 6 - 10 16-20 21 -25 more than 25

Average Time Spent Reviewing Teacher Candidate Portfolios When PortfoUos Were Submitted (n = 11)

1-5 minutes 6-10 minutes 11-15 minutes 16-20 minutes more than 20 minutes

When Teacher Candidate Portfolios Are Reviewed (n = 11) Before the interview During the interview After the interview In Ueu of an interview

How Teacher Candidate Portfolios Are Reviewed (n = 11) Alone With coUeagues With the interviewee

Amount of Time Willing to Spend Reviewing Portfolios of Those Not Having Portfolios Submitted (n = 9)

1 5 minutes 6 10 minutes 11-15 minutes 16-20 minutes more than 20 minutes

n '

13

5

3

10 2 4 1

4

1 4 2

4

11 3 4

8 8 4

3 2 2 2

%

62

24

14

48 10 19 5

19

9 36 18

36

90 27 36

73 73 36

33 22 22 22

' Totals and percentages do not add to 100 because respondents could check more than one item and because of nonresponse and rounding.

75

Personnel administrators were asked to rank six items according to the value

placed on each during the hiring process. The items were ones normally requested or

presented during an appHcation or interview. Respondents used the following scale: 1

for most compelling item and 6 for least compelling item. The responses to the

question, reported by means, are presented in Table 4.10.

Table 4.10 Personnel Administrators' Ranking of Compelling Items in the Hiring Process, (n = 21)

Interview or Application Items

Teachmg Evaluations

Recommendation Letters

Transcript

Resimie

Written Philosophy

Sample Unit/Lesson Plans

Mean for AU

Respondents

2.14

3.29

3.38

3.81

3.86

4.52

Principals

n = 9

Mean

2.10

3.90

3.44

3.10

3.80

4.70

Superintendents

n = 6

Mean

2.69

2.67

3.50

3.33

3.83

5.00

Persormel Directors

n = 6

Mean

1.67

3.00

3.17

5.33

4.00

3.83

Research Ouestion 3

Research question 3: What types of portfolio evidences, based on the teaching

proficiencies for leamer-centered schools, authentically assess the performance of

home economics preservice teachersl

A list of 30 evidences, Hsted in Table 3.2, that could be used to assess each of

the five teaching proficiencies was included in the round-one researcher-developed

76

checkHst (see Appendix B). Survey participants were asked to check the evidences

that best assess each proficiency. The checklists were sent to 96 participants in round

one of the Delphi process. A total of 60 usable responses were returned.

In the fust round of the Delphi process, 13 evidences received 70% or higher

agreement among the three participant groups and were considered to have achieved

consensus. Evidences receiving less than 20% agreement were considered to have

failed to reach consensus and were omitted from the round-two checkHst.

Those evidences that received between 20% and 70% were retained and

resubmitted to the participant groups in round two to attempt consensus on additional

evidences. Table 4.11 lists the evidences for each proficiency that received 70% or

higher agreement or less than 20% agreement.

In round one, respondents were provided the opportunity to add additional

evidences that best assess each teaching proficiency. A total of 11 additional

evidences were suggested by round-one respondents and were included in the round-

two checklist. Each of the leamer-centered proficiencies had at least one evidence

added for round two with four added for Leamer-centered Knowledge, two for

Leamer-centered Instmction, two for Equhy hi ExceUence for All Leamers, two for

Leamer-centered Communication, and one for Leamer-centered Professional

Development. Evidences suggested by respondents in round one and added to the

round-two checklist were marked with an asterisk. The evidences added m round two

are categorized in Table 4.12.

77

Table 4.11 Evidences Achieving Consensus and Failing to Achieve Consensus m Round One of the Delphi Process.

Teaching Proficiency


Leamer- centered Instruction

Equity in ExceUence for AU Leamers


Leamer- centered Professional Development

Evidences Achieving 70% Consensus

Instructional Materials (Teacher Designed)


Creative Activity Instmctional Materials (Teacher

Designed) Unit/Lesson Plans



Pupils' Evaluations Unit/Lesson Plans

1


Membership in Professional Associations

Philosophy/Goals Statement Professional Development Activities

(I.e., Meetings Attended)

Evidences Achieving Less Than 20%

Evidence of Community Service Entrepreneurship Plan Public Relations Project

Achievement Awards/Certificates Entrepreneurship Plan ExCET Scores Leadership in Professional

Associations Letters of Reconmiendation Membership in Professional

Associations Resume Transcript

Achievement Awards/Certificates Entrepreneurship Plan Evidence of Computer Skills ExCET Scores Leadership in Professional

Associations Letters of Recommendation Membership in Professional

Associations Professional Development Activities

(i.e., Meetings Attended) PubUc Relations Project Research Project Resume Transcript

Entrepreneurship Plan ExCET Scores Photographs of Teaching Activities Research Project Samples of Pupils' Work Transcript


Description of Teaching Strategy Used Documentation of Field Experiences

Prior to Smdent Teaching ExCET Scores Photographs of Teaching Activities PupUs' Evaluations Samples of Pupils' Work Unit/Lesson Plans Video-audiotape of Lesson

No evidence achieved 70% consensus for Leamer-centered Communication.

78

Table 4.12 Evidences Suggested in Round One of the Delphi Process for Assessing Teaching Proficiencies.

TEACHING PROFICIENCY


Leamer-centered Instmction

Equity in ExceUence for AU

Leamers


Leamer-centered Professional

Development

SUGGESTED EVIDENCES

* AbiUty to use a variety of telecommunications

* Evidence of use of appropriate software

* Mentorships

* PupU projects as a result of lesson

* Evidence of collaboration on projects

* Evidence of knowledge of models of teaching,

learning styles, etc.

* Evidence of attendance at ARD meetings or

knowledge of lEPs

* Evidence of outside school activities during

smdent teaching

* Evidence of parental contact in smdent teaching

* Evidence of teaching experience in volunteer

efforts (i.e., church, community agency)

* Case smdy of ethical dilemma

After analyzing the responses to round one, the researcher prepared the

checkHsts for round two. The evidences retained in round two for each of the five

proficiencies are shown in Tables 4.13 and 4.14. The round-two instrument (see

Appendix C), which included evidences receiving between 20% and 70% consensus in

round one and the new evidences for each proficiency suggested in round one, was

sent to 60 round-one respondents; 38 responses (63%) were received.

79

Table 4.13 Evidences of Teaching Proficiencies 1, 2, and 3 Retained for Round Two of the Delphi Process.

Proficiency 1 - Leamer-centered Knowledge

Achievement Awards/ Certificates Class Projects Related to Teaching

Field Classroom Procedures/Discipline

Plan Creative Activity Evidence of Computer SkiUs Description of Teaching Strategy

Used Documentation of Field

Experiences Prior to Smdent Teaching


ExCET Scores FHA/HERO or 4-H Activities Log Leadership in Professional


Associations Philosophy/Goals Statement Photographs of Teaching Activities Professional Development

Activities (i.e., Meetings Attended)

Pupils' Evaluations Reflective Journal of Smdent

Teacher Research Project Resume Samples of Pupils' Work Self-assessment Checklist/Essay Smdent Teaching Evaluations

(from Supervising Teacher, University Supervisor, etc.)

Transcript Unit/Lesson Plans Video/Audiotape of Lesson

Proficiency 2 - Leamer-centered Instmction


Evidence of Community Service Evidence of Computer Skills Description of Teaching Strategy

Used Documentation of Field E3q)eriences

Prior to Student Teaching Evaluation Devices (Teacher

Designed) FHA/HERO or 4-H Activities

Log PhUosophy/Goals Statement Photographs of Teaching

Activities Professional Development

Activities (i.e.. Meetings Attended)

Public Relations Project Pupils' Evaluations Reflective Journal of Smdent

Teacher Research Project Samples of PupUs' Work Self-assessment CheckUst/Essay Student Teaching Evaluations


Video/Audiotape of Lesson

Proficiency 3 - Equity in Excellence for AU Leamers

Classroom Procedures/DiscipUne Plan

Creative Activity Evidence of Community Service Description of Teaching Strategy


E;q)eriences Prior to Student Teaching


FHA/HERO or 4-H Activities Log Philosophy/Goals Statement Photographs of Teaching

Activities Reflective Journal of Student

Teacher Samples of Pupils' Work Self-assessment CheckUst/Essay Student Teaching Evaluations



1

80

Table 4.14 Evidences for Proficiencies 4 and 5 Retained for Round Two of the Delphi Process.

Proficiency 4 - Leamer-centered Communication

Achievement Awards/Certificates Class Projects Related to Teaching

Field Classroom Procedures/Discipline

Plan Creative Activity Evidence of Community Service Evidence of Computer SkiUs Description of Teaching Sti ategy


Experiences Prior to Smdent Teaching


FHA/HERO or 4-H Activities Log Instmctional Materials (Teacher

Designed) Leadership in Professional


Associations Philosophy/Goals Statement Professional Development

Activities (i.e., Meetings Attended)

PubUc Relations Project Pupils' Evaluations Reflective Journal of Student

Teacher Resume Self-assessment Checklist/Essay Smdent Teaching Evaluations

(from Supervising TeachCT, University Supervisor, etc.)

Unit/Lesson Plans Video/Audiotape of Lesson

Proficiency 5 -Leamer-centered Professional Development

Achievement Awards/Certificates Classroom Procedures/DiscipUne

Plan Creative Activity Evidence of Community Service Evidence of Computer SkUls Entrepreneurship Plan Evaluation Devices (Teacher

Designed) FHA/HERO or 4-H Activities

Log Instructional Materials (Teacher

Designed) Letters of Recommendation Public Relations Project Reflective Journal of Smdent Teacher Research Project Resume Self-assessment CheckUst/Essay Student Teaching Evaluations

(from supervising Teacher, University Supervisor, etc.)

Transcript

81

No evidences under the proficiency Leamer-centered Communication received

70% consensus in either round one or round two. Evidence of Parent Contact in

Student Teaching was the evidence achieving highest agreement for the proficiency in

round two, receiving slightly over 60% agreement. In order to retain at least one

evidence under Leamer-centered Communication, the researcher decided to accept all

evidences that received 60% or higher consensus in round two.

Table 4.15 lists the 13 evidences that received at least 70% consensus in round

one, and Table 4.16 lists the 11 evidences that received at least 60% consensus in

round two. Four of the evidences suggested by round-one respondents were accepted

by respondents in round two of the Delphi process: (1) ability to use a variety of

telecommunications; (2) evidence of knowledge of models of teaching, learning styles;

(3) evidence of attendance at Admission, Review, and Dismissal (ARD) meetings or

knowledge of Individual Education Plans (lEPs); and (4) evidence of parental contact

in student teaching. The 24 evidences receiving consensus from rounds one and two

provided 17 different evidences used to develop the portfolio prototype.

82

o

V3 (Z) O O

42

Q

0) c O •a O

(U

O

U S o

.s

>

< (U O

c >

U-5

CO

t/)

« —

one]

ni

str

- 2

o -a c

e 1 ^ «2i <

"« S .^ 2 § «3 S o •— Q 3 < ^

Eco

r er

Ed

n =

1

3 S K H

v> v> «

• H " § S O 2 t - f^ S W -^ c « 75

o « X £

M

c

I! ^ <

3

Evi

denc

jn

cies

fi

ci(

Pro

^

c

^

c

5

c

fis

^

r-

m «"

a\ r~

>n —

W-l r-

u-i ^

wn r-

w^ •T

u CQ

J3

2 CQ

.1 -2?

a

o e

k 4

a

i-i 0) s j3

5

VO r J p , VO

VO r<^ ^ ....

OO OO VO <n

en —I — —.

vri o VO ON

m 00 ^ M ^ M

o o r- r~-

fS <s Tt r r

2 "u

00

1 CO 0)

O

CS r -VO VO

f ^ - ^ ^ •^ ^ ^

ON CV

r^ r^

<o in -" -"

o »n t ^ ON

TT O N

^ ^

o o P~ 00

t S 00 • ^ TT

ka

u

•s Pi t^ « a a !S 2 ^ S

1 « fa Q is-u U ^ D

c o

•a

•2 4J

8 ka U c 3

s

— r- r- r-r - VO VO i n

>o • * • * <s ^ " .— ^

Tf 00 C \ 00 OO VO r^ VO

VO po l o r^ — — 1 — —

VO P«- VO OO

?s m f^ r-.— ^ ^ ^

«s o o o r- r- p- r

f^ CS ( S CS • ^ • * T f T T

2 "u '£ OA

l | s ^ ^2

c4 C CO

•« a a a

2 .2 .1 w g

— a p 3

<

«° 8 c u

"u u

(S .3 j2 >» S * r f M

•3 ^ S " ^

M

§ •a .2

o U

c

? ha U c c3 3

VO — t ^ 00

VO r -

o >n ON o \

P~ OO — —

in m r- 00

u-j r-.—t ^—

o r-00 00

00 CS Tf W)

s o •a rt

1 ^ < —

S -2 O M

1 o

^ a ^ a 5

lli

•a J O

£

1 S 8 S U _0

P "3 e3 >

CS VO

f<^ ^ i ^

ON P^

m —"

m r-

m -^

CS r-

f<^ • *

* r f

c

s 4J

00

i o

f s 1

00

r-^

m ON

00 —

o 00

VO ^*

m 00

^* m

c/l _4J

< ^

1 s < Q -o

is o £

c o

o U

c u I

ka

CO c u

i o r--a D u 

<u o Z

83

00

C/3 IZl

o O

o H c 3 O

C/2

3 1/3 C <D c« C o U o

c >

<

o

(U

>

VO

W)

5 "53 t ;

1 S5^ i-i " t ! •— C

£ | ^ <

cfl e •— 2 •g « 9 o '-V Q = r~

ECO

D

erE

d n

= 1

4j ^ ^—'

§ S = f -

W5

trt «

•H-s _ S S f7 2 H —

1 2^ " E ^ o P. X £

v> C

a?

All

ienc

es

Evi(

fici

enci

es

o £

fe^

c

65

&:

B5

c:

55

:^

VO u->

m

— r~

CS

r-VO

00

VO VO

«

C

D 1 o 1

< ^ -

OA

o

ka 4J

C

8 ka 4J

3

VO 0 0 P -<n r- VO

>n r~ VO

P- Ov — p~ i n p-

en o CS

p- o o VO m l o

OO VO VO

OO — m VO VO VO

VO r o Tics CS CS

2 "u iJu

..a

u

^ 5

11 tS 3 i a U Ou, .ai, o § « « ^

U W D

VO i n

m

CJN i n

o

(^ VO

00

*>M

VO

CS

75

2

?

• *

i n VO

-

p-VO

00

• M

VO

en CS

ka

o c«

00 H

S .a 3 ««

3 o U (4-1

o

i > u

c o

•a o

i

ka

Si a 8 ka

 u

p-vo

VO

p« rr

00

r4 ON

-

VO VO

i n CS

U a

o

•-a C/f

o ^

I'i Is o . a 2 « o

•»a «J

a ^ -

r^ VO fn <n

m in

m r^ VO T t

^ 00

i n CS t ^ Ov

Ov ~

- ^ (T^ VO VO

m Tics CS

75 Q 0-

< z.

« o

c 5 i;? o ° ,2 U fiA <a-i P O O

• 8 ^ "> S S*

00

< ka

4) U

c

= p '"* c >% ^ •3 .3 s

p« VO

VO

i n VO

ZZ

00 m

p-»

r-i VO

Tics

2 c o U

Ok U

I1 —

c o • 3

.3

o U

U k.1

C

8 ka D

00 p~

p^

i n f^

VO

m 00

O

„-VO

CO CS

3 c '5 U

<

a u 3 > u

IS u <

•a c

_o '35 .^ o

£

= 1 8 f h "l^ « > CO Q

.a u m

84

A total of 24 evidences from the round-one and round-two checklists did not

achieve consensus on any of the leamer-centered teaching proficiencies. The list of

evidences not reaching consensus in ehher round one or round two are listed in Figure

4.2. Evidences marked with an asterisk were the evidences suggested by round-one

respondents and added to the round-two checklist.

Discussion

Background information and demographic characteristics of the three

participant groups were obtained through a researcher-designed questionnaue sent to

study participants during the fust round of a two-round Delphi process. Rounds one

and two of the Delphi provided consensus for 17 different evidences that authenticaUy

assess mastery of the five leamer-centered teaching proficiencies.

A brief discussion of fmdings for each of the first three research questions of

this study is given. Descriptive statistics were used to report the data in tables and

figures. The researcher considered using chi square to analyze differences among the

groups, but the numbers in the study were too small to provide adequate data for

complete analysis.

85

* Case Study of Ethical Dilemma


Description of Teachmg SUategy Used


Entrepreneurship Plan

* Evidence of Collaboration on Projects


* Evidence of Outside School Activities During Student Teachmg

* Evidence of Teachmg Experience m Volunteer Efforts

(i.e., church, community agency)

* Evidence of Use of Appropriate Software

ExCET Scores



* Mentorships

Photographs of Teachmg Activities

Public Relations Project

* Pupil Projects as a Result of Lesson


Research Project

Resume

Self-assessment Checklist/Essay

Student Teaching Evaluations (from Supervising Teacher, University Supervisor, etc.)

Transcript


* Evidences suggested by respondents to round one of the Delphi process.

Figure 4.2 Evidences Not Reaching Consensus for Any Proficiency in Rounds One and Two of the Delphi Process.

86

Research Ouestion 1

Research question 1: What are selected characteristics of this study's

participant groups: home economics teacher educators, home economics preservice

teachers, and Texas public school personnel administrators responsible for hiring

home economics teachers'}

The sample of Texas home economics preservice teachers (n = 40) represented

nine of the 12 Texas university home economics education programs. The small

number of home economics preservice teachers reflects a nationwide decline in the

number of students enrolling in home economics education programs.

Twenty-five percent of the respondents held bachelor's degrees in another field

of home economics and had retumed to the university for home economics teacher

certification through the post-baccalaureate program. Because none of the preservice

students was enrolled under the Jamison Bill, a certification program that credits work

experience in the teaching field toward college hours, the returning students either had

received their baccalaureate degree within the previous three years or had not held

employment positions in a home economics area following graduation. The number of

students returning for teacher certification at the post-baccalaureate level may reflect

the shortage of home economics teachers in Texas at the thne of this study.

Teaching experience in the pubhc school classroom could have been a factor in

the selection of evidences that assess mastery of the proficiencies. A comparison of

the number of years of public school teaching experience between home economics

teacher educators and personnel administrators revealed the foUowing: 84% of the

87

home economics teacher educators had 10 years or less of public school teaching

while 81% of the personnel administrators had 11 years or more of public school

teaching experience.

Limited experience in the supervision of classroom teachers could have been a

factor in the selection of evidences that assess mastery of the proficiencies. The

number of home economics preservice teachers supervised by home economics teacher

educators in field-based teaching experiences varied. Six of the 19 respondents

supervised no student teachers in the four-semester period between August 1993 and

May 1995, while two respondents supervised 29 or more during the same time.

Texas public school personnel administrators responding to the questionnaire

represented 16 of the 20 education service districts in Texas. The four districts not

represented by a respondent to the study were, except for the district with Austin as its

center, in far south and central southwest areas of the state. The districts with no

respondents represent a large land area but account for only 19% of the counties in

Texas and 15% of the pubHc school districts in Texas. Both large and smaU school

districts were represented.

The number of teacher candidates interviewed by the Texas public school

personnel administrators in the past school year varied widely. More personnel

administrators from small school districts responded to the study than did personnel

admhiistrators from large school districts. Sixty-two percent of the admuiistrators

responding to the questionnaire mterviewed 15 or less m the past school year. Three

88

(14%) of the administrators interviewed more than 76 teacher candidates in the past

year.

Research Ouestion 2

Research question 2: How are portfolios currently being used by home

economics teacher educators, home economics preservice teachers, and Texas public

school personnel administrators who are responsible for hiring home economics

teachers'}

Although 65% of the home economics preservice teachers had developed a

portfoHo in either a course or in student teaching, only 23% of those had used a

portfoHo in a job interview. Preservice teachers may not have been instructed about

the potential of portfolios in the hiring process, or they may not have had the

opportunity to use a portfolio in interviews. The review of Hterature suggests that

portfolios are helpful in seeking employment (Grady, 1992; Feuer & Fulton, 1992;

Bouas, Bush, & Fero, 1994).

Though the number of preservice teachers developmg portfolios as a course

requirement (n = 10) was similar to the number developing portfolios during student

teaching {n = 9), the portfolios developed for courses or student teaching may not be

thought suitable for employment purposes, hi research by Snyder, ElHott, Bhavnagri,

and Boyer (1993), public school and univershy teachers and administrators reviewed

portfolios and judged them adequate representation of preservice teachers' work but

concluded that portfolios for hiring may need to be constructed hi a different manner.

89

Home economics preservice teachers and home economics teacher educators

viewed the primary purpose of developing a professional portfolio as a tool to market

the preservice teacher's competencies. Teacher educators and preservice teachers both

ranked proficiency in teaching as the second most important reason for developmg a

portfolio.

Home economics preservice teachers who had developed a portfolio ranked to

demonstrate personal development second, while those who had never developed a

portfolio ranked it fourth. This finding is consistent with previous research studies on

portfoHo development and use, which indicated that those developing portfolios

reported an increase of personal insight and growth (Ohlhausen & Ford, 1990; Snyder,

EUiott, Bhavnagri, & Boyer, 1993).

Fifty-two percent of personnel administrators had reviewed teacher candidates'

portfolios in the past year. The time spent reviewing teacher portfolios varied from

less than five minutes to more than 20 minutes on each portfolio. The majority of

respondents (63%) spent less than 15 minutes per portfolio. This fmding supports in a

study by Smolen and Newman (1992) where 80% of the administrators said they were

willing to spend 3-15 minutes reviewing each portfolio.

Of the personnel administrators in this study who had never reviewed a teacher

candidate's portfolio, 44% were willing to spend more than 15 minutes on each

portfoHo submitted. While this fmding reflects a poshive attitude toward the use of

portfolios in the hiring process, the normal schedule of principals, superintendents, and

personnel directors may not aUow time to review each portfoHo.

90

While 27% of the personnel administrator reviewed the portfolio during the

interview, only 36% reviewed the portfolio with the teacher candidate. The purpose

and importance of each piece of evidence in the portfolio must be clear to the

reviewer. If the portfolio is reviewed without the candidate present, reflective

statements included as an integral part of the portfolio can alert the reviewer to how

each piece of evidence illustrates the teaching proficiencies. Green and Smyser (1996)

noted that reflective statements integrate all parts of the teaching experience. None of

the persormel administrators in this study reviewed portfolios in lieu of an interview,

although a study by Newman, Smolen, and Newman (1993) found administrators

willing to hire without an interview if an error-free portfolio were presented along

with a coUege transcript and a videotape of actual teaching.

Personnel administrators were provided a list of six items usually presented in

an application or during an interview and asked to rank them from most compelling to

least compelling in the hiring process. Of the six items, all respondents ranked

teaching evaluations and letters of recommendation as most compelling; least

compelling was sample unit/lesson plans.

In a study by Smolen and Newman (1992), admhiistrators found resumes,

letters of reconunendation, and student teaching evaluations to be essential portfolio

hems; unit plans were of little mterest. Admhiistrators participatmg m a study by

Bouas, Bush, and Fero (1994) rated resume, certification, and univershy placement

files (transcript and letter of reconunendation) as most helpful but were evenly divided

on the helpfulness of the unit and lesson plans.

91

The job responsibility of the personnel administrator may influence how

evidences in a portfoHo are valued. Personnel duectors m this study viewed teaching

evaluations and unit/lesson plans as more compeUmg than did principals and

superintendents; principals and superintendents rated resume as more compelling than

did personnel dkectors. In a study by Smolen and Newman (1992), prmcipals wanted

to see more concrete evidence of teaching skills (unit and lesson plans, evidence of

computer skiUs, classroom management plan) than did supermtendents or personnel

dh-ectors. Because more personnel dkectors and superintendents responded to this

question, the resuhs may not represent the hems that are most compeUmg to

principals.

Research Ouestion 3

Research question 3: What types of portfolio evidences, based on the learner-

centered teaching proficiencies, authentically assess the performance of home

economics preservice teachers'}

A two-round Delphi process was used to reach consensus on evidences that

authentically assess mastery of the five leamer-centered teaching proficiencies. The

five proficiencies address knowledge, instruction, equity in excellence for all leamers,

communication, and professional development.

Thirty suggested evidences were submitted to study participants in the fust

round of the Delphi process. Thirteen evidences, representing four leamer-centered

proficiencies, achieved 70% consensus in roimd one. No evidences reached 70%

92

consensus for the proficiency Learner-centered Communication. Eleven additional

evidences were suggested by round-one respondents, and these were included m the

second round of the Delphi process (see Table 4.12).

Nearly all evidences used m tiie initial checklist of the Delphi process, as well

as evidences suggested by round-one Delphi process respondents and used in the

round-two checklist, were comparable to contents of teachmg portfoHos found m the

review of Hterature. Evidences such as FHA/HERO or 4-H Activities Log,

Entrepreneurship Plan, and ExCET Scores which are specific to Texas home

economics programs, were not found m the Hterature; however, references to

extracurricular activities, achievement/test scores, and class projects were found in

portfoHo content Hsts.

None of the evidences received 70% consensus in round two for any of the

five proficiencies. The decision was made to accept all evidences with 60% or greater

consensus in round two. Of the 11 evidences achieving 60% consensus in round two,

four were evidences suggested by round-one respondents and added for round two of

the Delphi process. Three evidences achieved consensus for three different teaching

proficiencies: Instructional Materials (Teacher Designed), Class Projects Related to

Teaching Field, and Unit/Lesson Plans. Evidence of Computer SkiUs was selected for

two different proficiencies (see Tables 4.15 and 4.16).

Of the 24 evidences not achieving consensus on any of the five teaching

proficiencies, notable to the researcher were resume, transcript, student teaching

evaluations, and letters of recommendation. In the round-one questionnaire, personnel

93

administrators had listed resume, transcript, and letters of recommendation as the most

compelling items in an application or interview. Studies by Bouas, Bush, and Fero

(1994) and Smolen and Newman (1992) reached shnilar conclusions. These three

items are traditional items of assessment, particularly for the hiring process.

Summary

This chapter presented the fmdings of the fust three questions of this study.

The participants for the two-round Delphi process consisted of three groups who are

considered stakeholders: home economics preservice teachers, home economics


responsible for hiring home econonucs teachers. A total of 17 different evidences of

the five leamer-centered teaching proficiencies for Texas educators reaching consensus

in the two-round Delphi process were used to guide the development of the portfolio

prototype which is described in Chapter V.

94

CHAPTER V

DEVELOPMENT OF A PORTFOLIO PROTOTYPE


mastery of the five leamer-centered teaching proficiencies adopted for Texas

educators, and to develop a portfolio prototype, based on the identified evidences, for

assessment of home econonucs preservice teachers. The evidences of the proficiencies

were selected by consensus of three stakeholder groups: home economics preservice


administrators who are responsible for luring home economics teachers. The five

proficiencies outlined in Learner-Centered Schools for Texas (1994) are performance-

based standards for preparing educators and serve as an accountability system for

educator preparation programs in Texas.

This chapter describes the development of a portfolio prototype based on the

evidences that achieved consensus in a two-round Delphi process as reported in

Chapter FV. Though no prototypes of portfolios were discovered to guide the study,

the review of literature provided a framework for decisions regardhig the content and

organization of the prototype.

Procedures to validate the usefulness of the portfolio prototype in assessing

mastery of the five leamer-centered teaching proficiencies are outlined. The findings

of research question 4, which asked if a panel of experts would judge the researcher-

developed portfolio prototype useful, also are discussed.

95

Portfolio Prototype Development

The 1994 adoption by the Texas State Board of Education of five leamer-

centered teachmg proficiencies for educators in Texas served as the hnpetus for this

study. In an effort to guide home economics teacher preparation programs in Texas,

this research considered the portfolio as an ahemative assessment method to

effectively assess mastery of the five teaching proficiencies.

Using the evidences achieving consensus in the Delphi process, a portfolio

prototype was developed by the researcher. Research question 3, regarding the

selection of evidences, was discussed in Chapter FV. The following discussion of

portfolios, evidences of the teaching proficiencies, and portfolio organization provides

a framework for the development of the prototype.

Portfolios: An Authentic Altemative Assessment

In accord with the national trend toward altemative assessment methods, the

researcher considered a variety of altemative assessment strategies from the review of

Hterature. Altemative assessments are viewed as any strategy different from the

traditional paper and pencil tests, such as performances, products, and portfolios.

PortfoHos also are considered an authentic assessment strategy (Betts, 1994; Worthen,

1993). To be considered authentic, the assessment must have meaning to the student

(i.e., can be related to a clearly defmed objective or to an experience with which the

student is familiar), be rooted in real-life situations (Meyer, 1992; Arter, 1993), utiHze

a wide array of lifelike performance tasks (Popham, 1993), and allow the student to be

96

mvolved m the assessment process (Grady, 1992). Meyer (1992) further asserted that

an authentic assessment must put the student in control of thning, topic, pacmg, and

conditions. Portfolios appeared to meet each of the conditions described in the

literature as necessary for authentic assessment.

Stiggins and Conklin (1992) maintained that authentic assessments can be used

to demonstrate mastery of proficiencies. Portfolios, therefore, were chosen as the

method for authentically assessing mastery of the five leamer-centered teaching

proficiencies.

The Teaching Portfolio. This study focused on the teaching portfolio as an

authentic means of obtaining baseline data (French, 1992) and coUecting continuing

evidences of the extent to which preservice students meet the leamer-centered

proficiencies established for Texas educators. A teaching portfolio is a tangible record

of experience that shows progress from the beginning of the preservice program

through student teaching and beyond and is effective in assessing student progress

(Cramer, 1993; ReUly et al., 1993).

The Selective Portfolio. Courts and Mclnemey (1993) describe a selective

portfolio as a selection of items from a larger body of collected work. The selective

portfolio helps students to focus on significant and meanhigful aspects of leamhig, to

reflect on the relevance of theh- learning, and to make connections between theu

learning and real-Hfe.

The teaching portfolio created for this research and development study is a

selective type. A decision was made to choose items for inclusion in the portfolio

97

prototype based on the evidences selected by home economics teachers educators,

home econonucs preservice teachers, and personnel administrators through a two-round

Delphi process.

The Reflective Portfolio. A key factor in portfolio development is that as

students create a portfolio, they must analyze or evaluate theu own work. This

reflection process includes focusing on why the evidence was included in the portfolio,

how the evidence iUustrates a selected criterion, or what the selected piece of evidence

means to the professional or personal development of the creator.

According to Green and Smyser (1996), reflection allows the writer to weave

together all parts of the complex teaching experience. One or more reflective pieces

can be included in the portfolio (Polin, 1991; Wolf, 1989) or each entry may have a

reflection or evaluation comment attached (Feuer & Fulton, 1992). One outcome of

the reflective process used in developing portfolios is that students are exposed to

metacognition (the process of thinking about their thinking processes) which enables

them to become independent leamers and skilled self-evaluators. Reflective statements

are included as a major element of the portfolio prototype developed for this study

(see Appendix D).

Evidences of Leamer-centered Teaching Proficiencies

An initial Hst of 30 evidences (see Figure 3.1) that could be used to assess

mastery of the five leamer-centered teaching proficiencies was gamered from a review

of literature (Shackelford, 1993; Seldm, 1993; Bouas, Bush, & Fero, 1994) and fi-om

98

suggestions of home economics teacher educators. Eleven additional evidences were

suggested by respondents to round one of the Delphi process. Following two rounds

of the Delphi process, 24 evidences achieving 70% consensus in round one and 60%

or greater in round two provided the researcher whh 17 different evidences for

assessing mastery of the five leamer-centered proficiencies (see Figure 5.1).

Portfolio Organization

Teaching portfolios can be organized in a variety of ways depending on the

stated purpose. In determining the organizational format of the prototype, the

researcher considered several models of portfolio organization noted in the review of

literature (see Table 2.2).

AU of the models included examples or products of the teaching experience. In

the models reviewed, portfoHo components related to professional development

(Mathies & Uphoff, 1992; Scriven, 1991; Jensen & Shepston, 1992; Wolf, 1991),

reflection (KiUion & Todham, 1991), and materials from others (Seldin, 1993; Barton

& CoUins, 1993). The organizational model proposed by Seldin, which included

material from oneself, material from others, and products of good teaching, was

selected as the model for this study's portfolio prototype because of its simplicity and

its documented success in a variety of settings.

99

Ability to Use a Variety of Telecommunications



Creative Activity

Evaluation Materials (Teacher Designed)

Evidence of Attendance at ARD Meetings or Knowledge of lEPs

Evidence of Computer SkUls

Evidence of Knowledge of Models of Teaching, Learning Styles, etc.

Evidence of Parent Contact in Student Teaching





Professional Development Activities (i.e.. Meetings Attended)

PupUs' Evaluations

Samples of PupUs' Work

Unit/Lesson Plans

Figure 5.1 Evidences of the Teaching Proficiencies Used as a Basis for a PortfoHo Prototype.

The Portfolio Prototype

The development of a portfoHo prototype was based on the evidences that

achieved consensus in a two-round Delphi process. The review of Hterature provided

a framework for decisions made regarding the content and organization of the

prototype.

Prototype Organization. The tities of the organizational components of Seldin's

portfoHo model: materials from oneself, materials from others, and products of good

100

teaching, were modified in this study. Evidences of the teaching proficiencies

reaching the selected consensus levels on rounds one and two of the Delphi process

were categorized into divisions simUar to Seldin's components and were titled

Background Information, Information from Others, and Products of Teaching. Table

5.1 shows the evidences included under each of the three divisions of the portfolio

prototype.

Prototype Content. Home economics teacher educators and home economics

preservice teachers were asked during round two of the Delphi process to contribute

examples of evidences for the portfolio prototype. Examples of evidences could be

class assignments and/or products of teaching. Using the contributed examples as a

base, the researcher developed examples for each of the 17 evidences which had

achieved consensus of the three stakeholder groups in the two-round Delphi process.

At least one example of each evidence that reached consensus in the two-round Delphi

was included in the prototype. Contributed examples of evidences were reformatted to

maintain anonymity.

Because neither a resume nor a transcript was selected as evidences to assess

mastery of the teaching proficiencies reaching consensus, an introductory section was

developed to introduce the portfolio developer to the reviewer. Three portfolio

organization models served as a basis for this decision: Green and Smyser (1996)

included both an introduction and an influences section; Seldin (1993) included a

section on materials from oneself; and Wolf (1991) included a section on background

information.

101

Table 5.1 Portfolio Prototype Organization.

Portfolio Prototype Division Examples of Evidence

Background Information Achievement Awards/Certificates




Professional Development Activities

(i.e., Meetmgs Attended)

Information from Others Pupils' Evaluations

Products of Teaching Ability to Use a Variety of Telecommunications


Creative Activity

Evaluation Materials (Teacher Developed)

Evidence of Attendance at ARD Meetings or

Knowledge of lEPs


Evidence of Knowledge of Models of Teaching,

Learning Styles, etc.

Evidence of Parent Contact in Student Teachmg

Instmctional Materials (Teacher Developed)


Unit/Lesson Plans

102

Reflective statements were added by the researcher for most examples of

evidence. Reflective statements provide reason for inclusion of the item in the

portfolio and explain the hnportance of the hem to the teaching process (Green &

Smyser, 1996; PoHn, 1993; Wolf, 1989).

Based on the Hterature, the researcher judged the following evidences of the

five leamer-centered proficiencies selected by the three participant groups as products

of teaching: instmction and evaluation materials, unit/lesson plans, evidence of

computer skiUs, and evaluations from students. Products of teaching may be used as

separate assessment methods (Perrone, 1991; Fisher, 1993) or may be mcluded in

portfolios (Arter, 1993; Meyer, 1992).

The researcher considered leadership in professional associations and

professional development activities as performance-based evidences, where students

demonstrate a skiU or perform a series of tasks (Wiggins, 1992; Raybom, 1993).

Other evidences that were considered performance-based by the researcher were

attendance at Adnussion, Review, and Dismissal (ARD) Meetings for special

population students and parent contact.

A Table of Contents Hsted the examples of evidences included in each of the

three divisions of the portfolio prototype: background information, information from

others, and products of teaching. Examples of evidences were grouped in subdivisions

within each main division. A page entitled Organization of Portfolio aligned the

examples of the evidences developed for the portfoHo prototype with the five leamer-

103

centered teaching proficiencies. Figure 5.2 shows the alignment of the teaching

proficiencies with the examples of the evidences.

Evaluations from students was the only evidence reaching consensus that

appeared to fit in the section Information from Others. Because several examples of

letters from students had been contributed by home economics preservice teachers, the

researcher made the decision to mclude examples of the students' letters m this

section.

Portfolio Presentation. Several general guidelines regarding portfolio

development were considered: the portfolio prototype should be logically organized,

should allow the viewer to move easily through and between divisions of the portfolio,

and should be visuaUy pleasing. Reviewers of the portfolio prototype should be able

to easily identify the evidences that were intended to assess mastery of each leamer-

centered teaching proficiency. The portfolio prototype was professionaUy presented in

a three-ring binder with clearly labeled divider pages. Colored pictures and page

borders provided added interest.

Selected examples from the portfolio prototype are in Appendix D including

the cover, table of contents, division pages, and explanation pages for each division.

Examples of evidence from each division also are provided.

104

ORGANIZATION OF PORTFOLIO

The evidences presented m the portfolio are ahgned with the teaching proficiencies outlmed m Learner-Centered Schools for Texas: A Vision of Texas Educators (Texas Education Agency, 1994) in the foUowmg manner:

• Leamer-centered Knowledge Introduction

Rationale for Teaching as a Career Career Goals Philosophy of Teaching Home Economics

University Class Projects Related to Teaching Field Technology Projects Professional Activities Lesson Plans

• Leamer-centered Instruction University Class Projects Related to Teaching Field Evaluations from High School Students Block Plan Lesson Plans Teacher-developed Materials Technology Projects Creative Activities Special Needs Students

• Equity in Excellence for All Leamers Evaluations from High School Smdents Special Needs Smdents Sample of Smdents' Work 4MAT Lesson Plan Technology Projects

• Leamer-centered Communication Parent Contacts Technology Projects Letters from High School Smdents Block/Lesson Plans

• Leamer-centered Professional Development Professional Activities Honors and Awards Certifications Career Goals

Figure 5.2 Alignment of PortfoHo Prototype Evidence Examples with Teaching Proficiencies

105

Procedures for Validation of the Portfolio Prototype

The portfolio prototype was presented to a panel of experts as a means of

answering research question 4 (usefulness of the portfolio). The panel was

comprised of members comparable to the three groups used in rounds one and two of

the Delphi process: home economics preservice teachers, home economics teacher

educators, and Texas public school personnel administrators who are responsible for

hiring home economics teachers.

In round two of the Delphi process, a response card (see Appendix A) was

included; respondents were asked if they would be willing to serve on a panel to

evaluate the portfolio prototype. Twenty-seven cards were retumed with 19

respondents agreeing to serve. From the 19 cards agreeing to serve on the panel of

experts, four response cards were randomly selected from each of participant groups.

Telephone caUs were made to each of those selected to determine their willingness to

review the completed portfolio prototype.

The four home economics preservice teachers and the four home economics

teacher educators drawn from the response cards agreed to serve on the panel. The

four home econonucs teacher educators on this researchers' dissertation committee also

agreed to review the portfolio prototype.

Two of the four personnel administrators who retumed reply cards had

accepted positions with new school districts and declined to serve on the review panel.

The researcher contacted two Texas public school personnel administrators who had

not been a part of the origmal participant group and who represented school districts

106

of different sizes from the two remaining administrators drawn from the response

cards. Both personnel administrators agreed to serve on the panel.

A copy of the researcher-designed portfolio prototype was sent to each person

on the 16-member panel along with a three-page questionnaue (Appendix E). The

fust page of the questionnaire provided an overview of the portfolio prototype review

process. The second page of the questionnaire was a matrix-style rating sheet that

asked reviewers to rate the usefulness of each group of evidences included in the

portfolio prototype against the five teaching proficiencies. A five-point numeric rating

scale from 1 (of no use) to 5 (very useful) was used.

The panel was asked to evaluate the items included in the portfoHo prototype

only on the degree to which the examples of each evidence are useful for assessing

mastery of a teaching proficiency. Fifteen usable questionnaires were retumed.

Responses to the matrix-style rating sheet were analyzed, and means for all

respondents and for each group of respondents were calculated (see Appendix F).

A third page of the questionnaire contained stmctured questions that asked the

panel to provide information regarding the overall usefulness of the portfolio

prototype. Open-ended questions aUowed reviewers to suggest improvements in the

portfolio prototype content and stmcture. Specific suggestions for unprovement were

categorized through content analyses.

107

Research Question 4

Research question 4: Wdl a panel of experts judge a portfolio prototype -

based on evidences selected by home economics teacher educators, home economics

preservice teachers, and Texas public school personnel administrators who are

responsible for hiring home economics teachers - useful for assessing mastery of the

five learner-centered teaching proficiencies?

For the purposes of discussion of findings for research question 4, the examples

of evidences developed by the researcher for the portfolio prototype will be called

examples to clearly distinguish them from the evidences reaching consensus through

the Delphi process (see Chapter IV). Mean scores for examples chosen by each group

in the review panel are presented in Appendix F. Because of the small numbers in the

data for this research question and because the data includes responses from four home

economics teacher educators who also served on the researcher's dissertation

committee, caution is advised in mterpreting the fmdings.

In general, persormel administrators rated examples higher for usability in

assessing each proficiency than did either home econonucs preservice teachers or

home economics teacher educators. Preservice teachers tended to rate the examples

somewhat higher than did home econonucs teacher educators.

On four of the five proficiencies, personnel administrators rated Parent Contacts

over one point higher than did the other two panel groups. Home economics teacher

educators and preservice teachers may not be as aware of the increased emphasis on

108

parent contacts as an important component in public school discipline and evaluation

programs.

A brief summary of the fmdings is provided for each proficiency. The two

highest rated examples in the portfolio prototype and the two lowest rated examples

for each proficiency are highHghted (see Table 5.2).

1. For the proficiency Leamer-centered Knowledge, Uiut/Lesson Plans and

Teacher-developed Instmctional Materials were the portfolio examples receiving the

highest means for aU respondents (M = 4.9; M = 4.82). Parent Contacts received the

lowest combined rating (M = 3.0); although personnel administrators rated Parent

Contacts high (M = 4.67).

2. For the proficiency Leamer-centered Instruction, Unit/Lesson Plans and

Technological Activities were rated highest (M = 4.83). The introduction to the

portfoHo rated lowest for this proficiency (M = 2.89), with Honors and Awards and

Parent Contacts each rating M = 3.0. Persormel administrators rated this example of

evidence higher than did superintendents.

3. For the proficiency Equity in Excellence for All Learners, Teacher-

developed Instmctional Materials and Teacher-developed Evaluation Materials rated

higher tiian other examples for this proficiency (M = 4.7; M = 4.75). Honors and

Awards and Professional Activhies, mcluded m the portfoHo, rated lowest for tius

proficiency (M = 2.33; M = 2.44).

4. AU portfoHo prototype examples for the proficiency Learner-centered

Communication were rated lower than were evidences on other proficiencies.

109

Proficiency in communication may be more difficult to assess in a portfoHo format.

Examples of communication evidences in the portfoHo prototype may not have been as

clearly defmed. Parent Contacts and Teacher-developed Evaluation Materials were

rated the highest (M = 4.25; M = 4.13). Lowest rated examples for this proficiency

were Samples of Students' Work (M = 2.78) and Certifications (M = 2.89).

5. For the proficiency Leamer-centered Professional Development,

Professional Activities was rated highest (M = 4.82). Other examples in the

background information section of the portfolio prototype received high ratings.

Samples of Students' Work and Creative Activities were the lowest rated for this

proficiency (M = 3.11; M = 3.22).

PortfoHo prototype examples that received the highest rating across aU

proficiencies were Teacher-developed Evaluation Materials and Teacher-developed

Instmctional Materials (M = 4.39; M = 4.35). Prototype examples receiving the

lowest rating across aU proficiencies were Introduction and Letters from High School

Students (M = 3.39; M = 3.5)

110

Table 5.2 Usefulness of Examples of Evidence by Proficiencies ^

Portfolio

Evidence

Introduction

Philosophy of Teaching

Professional Activities

Honors and Awards

Certifications

University Class

Projects Related to

Teaching Field

Evaluations from High

School Smdents

Letters from High

School Smdents

Unit/Lesson Plans

Teacher-developed

Instmctional Materials

Teacher-developed

Evaluation Materials

Creative Activities

Technological Activities

Samples of Smdents'

Work

Parent Contacts

Working with Special

Needs Smdents

Leamer-

centered

Knowledge

3.55

3.9

3.55

4.1

4.5

4.23

4.36

3.55

4,9

4.82 :

4.6

4.7

4.67

4.44

3.0

4.3

Leamer-centered Teaching Proficiencies

scale: 1 (of no use) - 5 ( very useful)

Leamer-

centered

Instmction

2.89

3.67

3.11

3.0

3.8

4.5

4.36

3.45

4.83

4.62

4.75

4.77

iiiiiiiiiiiMiii

4.69

3.0

4.6

Equity in

Excellence for

All Leamers

2.56

3.22

2.44

2.33

2.78

3.33

3.8

3.67

4.44

4.78

4.75

4.27

3.6

3.9

3.9

4.67

Learner-

centered

Communication

3.78

3.89

3.7

3.33

2.89

3.6

3.36

3.27

3.75

3.89

A.n

3.56

3.73

2.78

iiSSSisigiggiisiiisgss: ^

3.55

Leamer-

centered

Professional

Development

4.17

4.27

||i:;|||i|il||| iiiiiiiiiiiiii

4.31

3.7

4.0

3.56

3.88

3.67

3.75

3.22

4.1

3.11

3.5

3.7

The two examples with the highest means for each proficiency are shaded.

I l l

The review panel was asked how they would use a portfolio sinular to the

prototype developed for this study. Figure 5.3 lists answers m descending order of

choice and provides suggested uses of portfolios.

As noted in the document Learner-Centered Schools for Texas (1994), the five

leamer-centered teaching proficiencies are to be used as ". . . performance criteria and

standards for entities delivering educator preparation programs (p. vii)." Of interest to

the researcher was the fact that program evaluation had the fewest responses.

As a summative evaluation of student teaching

As a basis for writing recommendations for teachers/student teachers

For assessing teacher proficiencies in my school/school district

For hiring teachers (review information before or during an mterview)

As formative evaluation of student teaching

To provide evidence of my own teaching proficiencies in an interview situation

With high school students (as a graded project)

With high school students (for personal development only)

For appraisal of my teaching skills

For my own personal development

As a final grade for a course

For program development

For hiring teachers (review information to hire without an interview)

For program evaluation

Figure 5.3 Suggested Uses of Portfolios

112

The review panel rated the overall usefulness of the portfolio prototype based

on the uses listed in the previous question (see Figure 5.3). On a five-point scale from

of no use to very useful, all respondents rated equally the portfolio prototype as useful

and very useful.

The review panel assessed the physical/functional characteristics of the

portfolio prototype. Table 5.3 shows responses to the checklist.

Table 5.3 Physical/Functional Characteristics of the Portfolio Prototype

Characteristics

Has adequate description of portfolio items

Has dividers clearly marked

Is easy to review

Is logically organized

Has a professional appearance

Won't easily come apart

(N = 12)

10

10

6

9

10

11

%

83

83

50

75

83

91

^ Numbers and percentages wiU not equal 100 because respondents could check all items and because of nonresponse and rounding.

Easy to review had the least positive response. Subsequent research efforts

might address this physical/functional characteristic of the prototype.

Specific suggestions for improvement of the portfolio prototype were

categorized through content analysis. Recurring themes were professional

presentation - " very professional presentation," "very attractive and professional," and

excess of material - "ahnost too exhaustive," "lots of reading but very informative."

113

When asked for specific suggestions for portfoHo prototype improvement,

respondents provided feedback related to three questions: items to add, items to omit,

and suggestions for changes in appearance and organization. Themes regarding items

to add were as follows: the desire for more specific documentation, for example,

resume, transcript, teaching certificate, letter of recommendation, evaluation from

professor or supervising teacher; and the need for additional visual items such as

pictures of interaction with students, examples of student projects, charts to consolidate

data, and video tape.

Respondents were asked to suggest which portfolio items could be omitted.

DupHcate items in the portfolio (i.e., lesson plans, samples of student work, and

teacher-developed teaching aids) and the portfoHo length were two areas mentioned.

Characteristic responses to the questions were to shorten the introduction or to replace

it with a resume and to limit the description on professional organizations. One

respondent suggested onuttmg "everything except phUosophy, unit/lesson plans, and

evaluation materials" in order to have a "more efficient, reaHstic tool."

Suggestions to hnprove the organization of the portfolio included using tabs on

the divider sheets or using divider sheets with an extended edge. One respondent

wrote, "I Hke leatiier-bound notebooks with a zipper [for portfoHos]." One respondent

suggested using different paper for the divider sheets.

114

Discussion and Summary

One purpose of this study was to develop a portfolio prototype for assessment

of home econonucs preservice teachers. The portfolio prototype was based on the

evidences of the five leamer-centered teaching proficiencies reached by consensus of

three stakeholder groups: home economics preservice teachers, home economics


responsible for hiring home economics teachers.

This chapter mcluded (1) a rationale for the selection of the portfolio as the

authentic ahemative method of assessmg mastery of the leamer-centered proficiencies;

(2) a synopsis of the Hterature on teaching portfolios and portfolio development as h

applied to creatmg the prototype, and (3) specific elements of the fmal portfoHo

including selection, reflection, and organization. Because no research-based portfolio

prototypes were located in the Hterature, the evidences selected in a two-round Delphi

process and Seldm's model (1993) of organizing a teaching portfolio laid the

foundation for the researcher-developed examples included in the prototype.

Findings for research question 4, regarding the usefulness of the portfolio

prototype, also were discussed in this chapter. Data from a panel of experts (teacher

educators, preservice teachers, and persormel admiiustrators) were analyzed using

quantitative methods, though the sample was small. C^iahtative methods were also

used and may be more defensible.

Responses to research question 4 must be balanced with the results of the

Delphi process, used in question 3, in which consensus was reached on specific

115

evidences. Though the panel of experts was asked to focus on usabilhy for research

question 4, responses indicated that h was difficult to view the examples at "face

value." Respondents tended to make judgments about the content of the examples or

question the validity of the evidence to assess mastery. The validhy question had been

dealt whh m the discussion of research question 3 (see Chapter IV).

Reaching consensus on the evidences that assess mastery of the five teaching

proficiencies was an objective process. The five leamer-centered teaching

proficiencies may require altemative assessments, such as portfolios, products, and

performances, to show mastery, thereby verifying the need for this study. The

challenge was to create a portfolio prototype with examples that acciuately reflect the

evidences.

A prototype was defmed in the operational definitions as an initial functioning

model of a proposed design. The term heuristic (trial and error) is often appHed to a

prototype. The portfoHo prototype for this study was the first model of a proposed

design that wUl be improved as preservice teachers create their individual portfolios.

The prototype framework has been established.

116

CHAPTER VI

SUMMARY, CONCLUSIONS, AND

RECOMMENDATIONS

This chapter includes a summary of the study and conclusions drawn from

analysis and hiterpretation of the data. Recommendations also are mcluded m this

chapter.

Summary of the Study


mastery of the five leamer-centered teaching proficiencies adopted for Texas educators

and to develop a portfolio prototype, based on the identified evidences, for assessment

of home economics preservice teachers. The evidences of the proficiencies were

selected by consensus of three stakeholder groups: home economics preservice


admiiustrators who are responsible for hiring home economics teachers.

The five teaching proficiencies, adopted by the Texas State Board of Education

in 1994 and published in Learner-Centered Schools for Texas: A Vision of Texas, are

performance-based standards for preparing educators and serve as an accountability

system for educator preparation programs in Texas. The five proficiencies address

areas of knowledge, instmction, communication, equity in excellence for all leamers,

and professional development.

117

With the new Texas proficiencies, the need for revised assessment strategies

was evident. Portfolios were one suggested means of assessing the teaching

proficiencies. No research was found that addressed the topic of teaching portfolios

for home econonucs preservice teachers.

A review of opinion papers and research Hteratiue revealed that assessment

reform across the nation focused on methods that provide closer relationships between

teaching, learning, and assessment and aUow aU consthuencies to clearly understand

and be involved in the assessment process. One theme in assessment reform is a

move from the tradhional pencil-and-paper and standardized testing to a greater

emphasis on the abiHty to put knowledge into practice through a performance or a

product.

The portfolio emerged as a form of altemative assessment that aUows the

student to take a stronger role in the learning/assessment process. Portfolios are

considered both authentic and performance assessment. Performance assessment

involves a demonstration of skills and knowledge reflecting real-life situations, while

authentic assessment requires that the student demonstrate knowledge and skills in a

real-life setting and be actively involved in the assessment process through selection of

evidences and timing of the assessment.

Reflection on the learning process provides insight into growth and

development both for the portfolio developer and for portfolio reviewers engaged in

the assessment process. Not only does the portfolio developer select his or her best

118

work to include in the portfolio; but the developer explains, analyzes, and reflects on

the pieces of evidence included.

Teaching portfoHos aUow teachers to take personal responsibiHty for their

professional development and offer teacher education programs a means of assessing

student achievement as well as program effectiveness. Goal-setting, improvement of

teaching practices, and increased reflection on the teaching/learning process are keys to

the teaching portfoHo. In summary, research studies reviewed focused on the teaching

portfolio for assessment, in the hiring process, and for teaching improvement.

The participants for this study were three distinct groups who are stakeholders

in providing quaUty education in Texas. The participants included home economics

preservice teachers who completed their student teaching in spring 1995 (n = 40), aU

home econonucs teacher educators employed in home economics teacher education

programs in Texas {n = 26), and a random sampling of Texas pubHc school personnel

administrators (n = 30) who are responsible for hiring home economics teachers.

The researcher-developed instruments for this study were (1) a questionnaue

developed for each of the three participant groups to coUect pertinent background data

including education level, employment status, and experience with and use of

portfoHos; (2) round-one and rotmd-two checkHsts, used in a two-roimd Delphi

process, to achieve consensus on evidences that authenticaUy assess mastery of the

five leamer-centered teaching proficiencies; and (3) a matrix-style rating sheet and

one-page questionnaire used to vaHdate the usefulness of the researcher-developed

portfoHo prototype.

119

A modified two-round Delphi process was utilized in obtaining a list of

evidences that authentically assess tiie five leamer-centered teaching proficiencies.

The checklists for the two-round Delphi were sent to study participants in May and

September 1995. A 63% retum was achieved for each round with an effective

response rate of 40%.

A questionnaire on background information accompanied round one of the

Delphi checkHst. Collecting characteristics of participant groups allowed the

researcher to describe the subjects, to determine generalizability of the data, and to

explain inconsistencies in the findings.

From an list of 30 initial evidences and 11 suggested evidences generated in

round one, the three participant groups reached consensus on 17 different evidences

for the five proficiencies. The selected evidences served as a basis for the content of

the researcher-developed portfolio prototype.

Based on Seldin's model of portfolio development, the evidences were

categorized into background information, information from others, and products of

teaching. Reflective statements were used to substantiate inclusion of the examples of

evidence. The resulting researcher-developed portfolio prototype was subnutted to a

panel of experts for evaluation of usefulness in assessing mastery of the leamer-

centered teaching proficiencies. The matrix-style rating sheet and questionnaue, which

accompanied the researcher-developed portfolio prototype, were mailed to a 16-

member panel of experts (eight teacher educators, four preservice teachers, and four

persormel admiiustrators) in summer 1996.

120

Responses to the instmments were analyzed using descriptive statistics

(frequencies, percentages, and means) and reported in tables. Content analyses were

used for open-ended hems.

Summary and Interpretation of the Findings

The three stakeholder groups participating in this study were home economics

teacher educators, home economics preservice teachers, and Texas public school

personnel administrators responsible for hiring home econonucs teachers. The

majority of home economics preservice teachers were undergraduates seeking

employment as home economics teachers in Texas. A smaUer number of preservice

teachers were students with one degree in a specialization area of home economics

who were returning for teacher certification.

Home economics teacher educators responding to this study had more years in

university teaching than they did in pubHc school teaching. There was a wide

discrepancy in the number of student teachers supervised in a four-semester period.

This finding reflects a difference in the sizes of home economics education units in

Texas universities as well as the declining enrolhnent in home economics education

programs throughout the state.

Personnel administrators, the majority of whom were male, represented large

and small public school districts m Texas and most areas of the state. All personnel

administrators held at least one master's degree and two held doctorates. The majority

of administrators responding to the survey indicated they had more years experience in

121

public school classroom teaching than they had in administrative positions. Wide

differences in the numbers of teacher candidates interviewed by persormel

administrators reflected the difference in school populations of the school districts

represented in the study; the majority of the persormel admirustrators were from small

school districts.

At the time of this study, few home economics teacher educators, home

economics preservice teachers, and Texas public school personnel administrators were

using portfolios. Of the home economics preservice teachers who had developed

portfolios for courses or during student teaching, few had used them in the interview

process.

Both home economics preservice teachers and home economics teacher

educators perceived portfolios more useful for marketing teaching skills than for

assessing teaching proficiencies. Home economics preservice teachers who had

developed portfoHos rated their use for personal growth and development higher than

did preservice teachers who had never developed a portfolio.

Personnel administrators who had never reviewed a teacher candidate's

portfoHo were willing to spend more time reviewing portfolios than were those

personnel administrators who had experience in reviewing teaching portfoHos.

Teaching portfoHos were usuaUy reviewed by the administrators prior to the interview

and seldom were reviewed with the teacher candidate.

Teaching evaluations and letters of recommendation were ranked by persormel

administrators as the most compeUing items of evidence presented during the interview

122

process. Personnel directors viewed teaching evaluations and unit/lesson plans as

more compelling than did principals and superintendents, whUe principals and

supermtendents rated resumes as more compeUing than did personnel du-ectors.

The three participant groups in this study reached consensus on 24 evidences

for the five leamer-centered teaching proficiencies. Seventeen different evidences

served as the basis for the portfolio prototype: abiHty to use a variety of

telecommurucations, achievement awards/certificates, class projects related to teaching

field, creative activhy, evaluation materials (teacher designed), evidence of attendance

at ARD meetings or knowledge of lEPs, evidence of computer skiUs, evidence of

knowledge of models of teaching and learning styles, evidence of parent contact in

student teaching, instmctional materials (teacher designed), leadership in professional

associations, membership in professional associations, philosophy/goals statement,

professional development activities (i.e., meetings attended), pupUs' evaluations,

samples of pupUs' work, and urut/lesson plans.

Three evidences achieved consensus for three of the teaching proficiencies;

instmctional materials (teacher-designed), class projects related to teaching field, and

unit/lesson plans were selected as evidences for leamer-centered knowledge, leamer-

centered instmction, and equity in exceUence for aU leamers. Evidence of computer

skUls reached consensus for two proficiencies: leamer-centered knowledge and

leamer-centered instmction. Four evidences suggested by round-one respondents

achieved consensus in round-two of the Delphi process: abiHty to use a variety of

telecommunications (leamer-centered knowledge), evidence of knowledge of models of

123

teachmg/leammg styles (leamer-centered mstmction), evidence of attendance at ARD

meetmgs or knowledge of lEPs (equhy m excellence for all leamers), and evidence of

parent contact m student teaching (leamer-centered commuiucation).

Participants had difficuhy reaching consensus on evidences for the proficiency

Leamer-centered Communication. No evidences m either round of the Delphi process

achieved the desfred 70% consensus. The decision by the researcher to accept

consensus at the 60% level for round two allowed one evidence (evidence of parent

contact m student teaching) to be selected for the proficiency.

Twenty-four evidences from the two-round combmed Hst of 41 evidences failed

to achieve consensus for any of the teaching proficiencies. Among these were the

more traditional evidences such as transcript, resume, letters of recommendation, and

teaching evaluations.

The portfolio prototype, developed for this study based on Seldin's

organizational model for portfolios and the evidences achieving consensus in a two-

round Delphi, was judged as useful by a panel of experts. Examples of evidences in

the portfolio prototype achieving highest means for usabUity across all proficiencies

were instmctional materials (teacher-designed) and evaluation materials (teacher-

designed).

The portfolio prototype was viewed as professional in appearance with clearly

labeled dividers and adequate description of the portfolio items. The

physical/functional characteristics rated lowest was easy to review. Suggestions for

124

improvement to the portfoHo prototype included using fewer examples of evidences

and eliminating dupHcation of examples.

In this study, the fmdings mdicated some confusion on the part of the

respondents between the evidences of the proficiencies, reached by consensus in the

two-round Delphi process, and the examples of the evidences m the portfoHo

prototype. More traditional evidences were not selected for assessing mastery of the

proficiencies but were expected to be mcluded m the portfolio. The respondents, for

example, did not reach consensus on resumes, transcripts, letters of recommendation,

and certifications; yet, in the review of the portfoHo prototype, those were the items

they said should be included. The respondents were strongly in agreement on teacher-

developed instmctional and evaluation materials as evidence of three of the

proficiencies: knowledge, instmction, and equity of excellence for aU leamers;

however, some respondents suggested deleting aU teacher-developed materials to

improve efficiency in reviewing.

The portfoHo prototype developed for this study was the first model of a

proposed design that may be improved through a trial and error process. The model is

oitiy one idea of appropriate examples to iUustrate the evidences.

The results of this study must be interpreted cautiously due to limitations of the

investigation. A low retum rate, a lack of input from high school principals who hire

home economics faculty, and the absence of one university's group of preservice

teachers limits generalizabiHty.

125

All home economics teacher educators in Texas were included in the study to

assure that the results would be generahzable to all home econonucs teacher education

programs m Texas. An effective retum rate of 65% from home econonucs teacher

was achieved.

Although appropriate sampling procedures were followed for selectmg

personnel administrators, the small sample size prohibhs making generalizations to

personnel administrators in the 1058 public school districts m Texas. Respondents to

the round-one questionnaue represented 80% of Texas Education Service Districts.

There were no respondents from four districts that have tradhionally high Hispaiuc

populations. High school principals who hire home econonucs faculty represented

ortiy 11% of the round-two respondents.

The group of home economics preservice teachers in this study represented

over 90% of the population of Texas home econonucs preservice teachers in Spring

1995. Because an entire urtiversify group of preservice teachers was not represented,

generalizations of the findings to all preservice teachers in the state may be linuted.

Rounds one and two of the Delphi process were mailed in May and September

1995. These months are especially busy times for students, teachers, and

administrators. During and between these two months are times when people seek

new positions, move to new locations, and begin new endeavors.

In the intervening months, some administrators moved to new school districts.

Despite personal telephone calls to the nonrespondents, 70% of the administrators lost

126

comnutment to the study. The conunitment of home econonucs teacher educators

remained high throughout the study.

The mailing addresses for home economics preservice teachers were gathered

from uruversity home economics programs in May 1995. Permanent home addresses

and telephone numbers were secured when possible in order to update mailing

addresses. The frrst-round response rate of 50% for preservice teachers was due, in

part, to the lack of new addresses for those who had graduated in May or August and

had moved. Response rates were generally high for preservice teachers if their

teachers (home economics teacher educators) were committed to the study.

Conclusions

Based upon analyses of the data and interpretation of the fmdings, the

following conclusions were drawn:

1. A portfolio prototype is an authentic altemative assessment method that can

show evidence of the teaching proficiencies.

2. The Delphi process can be used successfully to collect data from groups

separated geographicaUy, when thne is limited, and in order to reduce negative aspects

of group interaction. The anonynuty of the Delphi process enables participants to

consider ideas on merit only and not on personalhy or status.

3. Evidences which can be used to authentically assess the five teaching

proficiencies include instmctional and evaluation materials; class projects in teaching

field; creative activity; unit/lesson plans; students' evaluations; leadership/membership

127

in professional organizations; philosophy/goals statement; professional development

activities; use of a variety telecommunications; computer skills; knowledge of models

of teaching and leammg styles; attendance at ARD meetmgs or knowledge of lEPs for

special populations students; and parental contact in student teaching.

4. The researcher-developed portfolio prototype, based on 17 evidences

achieving consensus in a two-round Delphi process, was judged useful by the panel of

experts in this study for assessment of mastery of the teaching proficiencies.

5. It is relatively easy for diverse groups to achieve consensus on the

evidences that authentically assess mastery of the teaching proficiencies. It becomes

more difficult to synthesize the information in order to create a portfolio prototype that

accurately reflects the evidences reached through consensus.

6. Portfolios may not be the most appropriate tool for assessing mastery of all

proficiencies; rather, ExCET scores and transcripts may be more appropriate for

assessing mastery of leamer-centered knowledge and video tapes or other strategies

may be more appropriate for assessing mastery of leamer-centered communication.

7. Portfolios offer promise of an altemative assessment method for use by

preservice teachers in demonstrating mastery of the proficiencies and for home

economics teacher education programs in assessing both preservice teachers and

determining the effectiveness of their program.

128

Recommendations

Recommendations for revisions to the portfolio prototype, use of the portfolio

prototype, replication of the study, and additional studies on teaching portfolios are as

follows:

1. RepHcate the portion of the study on evidences of the five leamer-centered

teaching proficiencies using larger sample sizes to support the use of inferential

statistics. Compare responses of the three stakeholder groups.

2. Use the portfolio prototype as a model to guide home economics preservice

teachers in the development of teaching portfolios. Provide copies of the prototype to

home economics teacher educators and conduct inservice with teachers and teacher

educators on portfolios as a means of authentic assessment of preservice and inservice

home economics teachers. Gather additional information on usability of portfoHos

during the portfolio development process.

3. Revise the portfolio prototype based on fmdings from this study;

specifically, delete duplications and use examples for more than one proficiency where

appropriate. Improve the review process by (1) using divider tabs in the portfolio and

(2) revising the review questionnaue to ahgn examples of evidence with the

proficiencies for which they were developed. Submit the revised portfoHo prototype to

a larger review panel.

4. Apply quaHtative methods to the development of a teaching portfolio, i.e.,

observe preservice home economics teachers through the portfolio development

129

process. Report their reactions, successes, and challenges during the development

process.

5. Apply the research and development procedures used in this study to

develop a portfolio prototype for inservice teachers to meet the requirements of the

Texas Teacher Appraisal System.

6. RepHcate the study with samples representing subject areas other than home

economics to compare similarities/differences m the evidences selected to assess

mastery of the five leamer-centered proficiencies and/or to reach consensus for all

preservice teachers.

7. Apply the findings of this study to the assessment of national standards for

home econonucs preservice teachers and for teacher education programs and/or for

home economics content proficiencies in Texas.

8. Study the use of the teaching portfolio in the interview for a teaching

position, the usability of the teaching portfolio in the interview/hiring process, and the

perceived benefit of the teaching portfolio to Texas public school personnel

administrators who interview home economics teacher candidates.

9. Develop a longitudinal study on the development and use of the teaching

portfolio throughout a professional teaching career.

Summary

In 1995, when this study began, the teaching proficiencies for Texas educators,

on which this study is based, had just been adopted, and statewide efforts to develop

130

assessment methods for the proficiencies had not been initiated. The nationwide trend

for educational reform had begun to focus on assessment reform with Darling-

Hammond (1990) advocating assessment as the place to begin reform movements. As

a result of the fmdings in this study, the researcher concluded that assessment reform

must move hand-in-hand with curricular and program reforms to provide students and

teachers the widest opportunities to achieve balance in teaching and learning.

PortfoHos offer a variety of assessment opportuiuties for both teachers and

students; yet this study has raised questions about the use of portfolios as an authentic

method for assessing mastery of the five leamer-centered teaching proficiencies for

Texas educators. A framework for portfoHo development has been established through

this study, and groundwork for further study has been contributed.

131

SELECTED BIBLIOGRAPHY

America 2000: An education strategy. (1991). Washmgton, DC: U. S. Department of Education.

Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers (2nd ed.) San Francisco: Jossey-Bass Publishers.

Arter, J. (1993). Altemative assessment: Promise and perils. In J. Bamburg (Ed.), Assessment: How do we know what they know? Dubuque, lA: Kendall/Hunt PubHshing Company.

Arter, J., & Spandel, V. (1992). Using portfoHos of student work in instmction and assessment. Educational Measurement: Issues and Practice. 11. 36-44.

Ashehnan, P., & Lenhoff, R. (1994). Early chUdhood education. In M.E. Krught & D. GaUaro (Eds.), PortfoHo assessment: Application of portfoHo analysis. Lanham, MD: University Press of America, pp. 65-76.

Association for Supervision and Curriculum Development (ASCD). (1991). Performance Assessment (video series manual). Alexandria, VA: Author.

Barton, J., & Collins, A. Portfolios in teacher education. Journal of Teacher Education. 44(3). 200-210.

Betts, F. (1994). Creating an electroruc portfoHo. Workshop manual of ASCD Professional Development Institute, Orlando, FL, January 1994.

Borg, W. R., & GaU, M. D. (1989). Educational research: An mtroduction (5tii ed.). White Plains, NY: Longman, Inc.

Bouas, J., & Bush, B., & Fero, G. (1994). Professional portfolios for tiie beghuung teacher. Education. 115(2). 266-271.

Brandt, R. (1990, March). On assessment hi the arts: A conversation with Howard Gardner. Educational Leadership. 47(6). 24-29.

Broadbent, W. A. (1989). Planning resource aid. Honolulu, HI: Hawaii State Department of Education, Office of Duector for Vocational Education.

132

Chittenden, E. (1991). Authentic assessment, evaluation, and documentation of student performance. In V. Perrone (Ed.), Expandmg student assessment, (pp. 22-31) Alexandria, VA: Association for Supervision and Curriculum Development.

CoUuis, A. (1993). Performance-based assessment of biology teachers: Promises and pitfalls. Joumal of Research in Science Teaching. 30f9). pp. 1103-1120.

Courts, P. L., & Mclnemey, K. H. (1993). Assessment m higher education: Politics. pedagogy, and portfolios. Westport, CT: Praeger Publishers.

Cramer, S. R. (1993). Navigatmg the assessment maze with portfolios. The Clearing House. 67(2). 72-74.

Cyphert, F.R., & Gant, W.L. (1970). The Delphi technique: A tool for coUectmg ophuons m teacher education. Joumal of Teacher Education XXI(3). 417-425.

Darling-Hammond, L. (1990). Achievmg our goals: Superficial or stmctural reforms? Phi Delta Kappan. 72(31 286-295.

Darling-Hammond, L. (1991). The implication of testing poHcy for qualhy and equalhy. Phi Delta Kappan. 73(4). 220-225.

Dodge, B.J., & Clark, R.E. (1977). Research on tiie Delphi technique. Educational Technology XVn(4). 58-59.

Doyle, D. P. (1991). America 2000. Phi Delta Kappan. 73(3). 184-191.

Eason, S. (1992, January/Febmary). Power assessment and the Delphi process. A paper presented at the Armual Meeting of the Southwest Educational Research Association, Houston, TX.

Feuer, M. J., & Fulton, K. (1992). The many faces of performance assessment. Phi Delta Kappan. 74(6). 478.

Fisher, T. H. (1993). Perspectives on altemative assessment: What's happening nationaUy? In T. Vemetson (Ed.), Selected papers from the Spring 1993 Breivogel Conference at the Uruversity of Florida on altemative/portfolio assessment, (pp. 12-16). Sarubel, FL: Florida Educational Research and Development Council, Inc.

French, R. L. (1992). Portfolio assessment and LEP students. In Focus on evaluation and measurement: Proceedings of the National Research Symposium on Lhnited EngHsh Proficient Student Issues. Volumes 1 and 2. Washington, DC.

133

Funkhouser, C. W. (1994). Education in Texas: Policies. Practices, and Perspectives (7th ed.). Scottsdale, AZ: Gorsuch Scarisbrick, Publishers.

Gay, L. R. (1992). Educational research: Competencies for analysis and apphcation (4th ed.). New York: Merrill (MacnuUan Publishing Company).

Geltner, B. B. (1993, October). Integrating formative portfolio assessment, reflective practice and cogrutive coaching into preservice preparation. Paper presented at the Armual Convention of the Uruversity CouncU for Educational Administrators, Houston, TX.

Grady, E. (1992). The portfolio approach to assessment. Bloomington, IN: Phi Delta Kappan Educational Foundation.

Green, J. E., & Smyser, S. O. (1996). The Teacher Portfolio: A Strategy for Professional Development and Evaluation. Lancaster, PA: Technomic Publishing Company, Inc.

Haney, W. (1991). We must take care: Fitting assessments to function. In V. Perrone (Ed.), Expanding student assessment (pp. 142-163). Alexandria, VA: Association for Supervision and Curriculum Development.

Hartle, L., & DeHart, P. (1993). The effective use of portfolio assessment with preservice teacher education: The University of Florida's elementary PROTEACH program. In T. Vemetson (Ed.), Selected papers from the Spring 1993 Breivogel Conference at the University of Florida on altemative/portfolio assessment, (pp. 134-145). Sanibel, FL: Florida Educational Research and Development Council, Inc.

Heibert, E., & Calfee, R. (1989). Advancing acadenuc Hteracy through teachers' assessments. Educational Leadership. 46(7). 50-54.

Herman, J. L. (1992). What research tells us about good assessment. Education Leadership. 49(8). pp. 74-78.

Herman, J. L., Aschbacher, P. R., & Wmters, L. (1992). A practical guide to ahemative assessment. Alexandria, VA: Association for Supervision and Curriculum Development.

Huikle, D. E., Wiersma, W., & Jurs, S. G. (1988). Applied statistics for the behavioral sciences (4th ed.). Boston: Houghton Mifflm Company.

134

Institute for Performance-Based Accountability. (1994). Report to Commission on Standards for the Teaching Profession: Program approval components and institutional indicators. Austin, TX: Author.

Jensen, R. A., & Shepston, T. J. (1992). Combining professional development plans and portfolios for use in preservice teacher education: Early chUdhood and elementary education perspectives.

Leiberman, A. (1991). Accountability as a reform strategy. Phi Delta Kappan. 73(4). pp. 219-220.

Lmstone, H. A., & Turoff, M. (1975). The Delphi method: Techniques and applications. Reading, MA: Addison-Wesley.

Maeroff, G. I. (1991). Assessing altemative assessment. Phi Delta Kappan. 73(4). pp. 273-281.

Marzano, R. J. (1992). A different kind of classroom: Teaching with dimensions of learning. Alexandria, VA: Association for Supervision and Curriculum Development.

Marzano, R. J., Pickering, D., & McTighe, J. (1993). Assessing student outcomes: Performance assessment using the dimensions of learning model. Alexandria VA: Association for Supervision and CXuriculum Development.

Mathies, B., & Uphoff, J. K. (1992). The use of portfolio development in graduate programs. Paper presented at 1992 Armual Conference of American Association of CoUeges of Teacher Education.

Meyer, C. A. (1992). What's the difference between authentic and performance assessment? Educational Leadership. 49(8). 39-40.

MitcheU, R. (1992). Testmg for leamhig. New York: The Free Press.

Newman, C , Smolen, L., & Newman, I. (1993, Febmary). Admmistrative responses to portfoHos prepared by teacher candidates. Paper presented at the Armual Meeting of the Eastem Educational Research Association, Clearwater, FL.

North H. Q., & Pyke, D. L. (1969). Probes of the technological future. Harvard Business Review. 47(5). 68-82.

Nweke, W. C. (1991, November). What type of evidence is provided tiu-ough the portfolio assessment method? Paper presented at the Armual Meeting of the Mid-south Educational Research Association (20th), Lexington, KY.

135

OhUiausen, M. M., & Ford, M. P. (1990). Portfolio assessment in teacher education: A tale of two cities. Paper presented at the 1990 National Reading Conference, Mianu, FL.

O'Neil, J. (1992). Putting performance assessment to the test. Educational Leadership. 49(8). pp. 14-19.

Paulson, F. L., & Paulson, P. R. (1990). How do portfolios measure up? A cognitive model for assessing portfolios. Paper presented at the Annual Meeting of the Northwest Evaluation Association, Uruon, WA.

Paulson, F. L., Paulson, P. R., & Meyer, C. A. (1991). What makes a portfolio a portfolio? Educational Leadership 48(5). pp. 60-61.

Perrone, V. (1991). Toward more powerful assessment. In V. Perrone (Ed.), Expanding student assessment (pp. 164-166). Alexandria, VA: Association for Supervision and Curriculum Development.

Polin, L. (1991). Portfolio assessment: The writing notebook. Eugene, OR: Visions for Leammg.

Popham, W. J. (1993). Circumventing the high costs of authentic assessment. Phi Delta Kappan. 74(6). pp. 470-473.

Raybom, R. (1993). Alternatives for assessing student achievement: Let me count the ways. In J. Bamburg (Ed.), Assessment: How do we know what they know? (pp. 24-30). Dubuque, lA: Kendall/Hunt Publishing Company.

ReUly, B., Hull, G., & Greenleaf, C. (1993). CoUaborative readings of hypermedia cases: A report on the development and testing of electroruc portfolios to encourage inquiry in teacher education. Joumal of Technology and Teacher Education. 1(1). 81-102.

Rich, S. (1994). Test me, test me not: The portfolio altemative for developmental writers. In M. E. Knight & D. GaUaro (Eds.), Portfolio assessment: Applications of portfolio analysis, pp. 47-63.

Rousculp, E. E., & Maring, G. H. (1992). Portfolios for a community of leamers. Joumal of Readmg. 35(5). 378-385.

Sackman, H. (1975). Delphi critique: Expert opmion. forecastmg. and group process. Lexington, MA: Lexington Books.

136

Scriven, M. (1988). Evaluatmg teachers as professionals. Unpublished manuscript. (ERIC Document Reproduction Service No. ED 300 882)

Secretary's Conunission on Achievmg Necessary Skills (SCANS). (1991). What work requnes of schools: A SCANS report for America 2000. U.S. Department of Labor: Author.

Seldin, P. (1993). Successful use of teaching portfolios. Bolton, MA: Anker Publishing Company, Inc.

Shackelford, R. (1993, Febmary). Useful materials to help others commurucate theu teaching accomplishments. Paper presented at the National Symposium on Improvmg Teaching Qualhy, San Antonio, TX.

Shuhnan, Lee S. (1987). Assessment for teaching: An irutiative for the profession. Phi Deha Kappan. (69), 38-44.

Simmons, R. (1994). The horse before the cart: Assessing for understanding. Educational Leadership. 51(5). 22-23.

Simmons, R., & Resruck, L. (1993). Assessment as the catalyst of school reform. Educational Leadership. 50(5). 11-15.

Smolen, L., & Newman, C. (1992, March). Portfolios: An estunate of theh validhy and practicality. Paper presented at the Armual Meeting of the Eastem Educational Research Association (15th), Hilton Head, SC.

Snyder, J., Elliott, S., Bhavnagri, N. P., & Boyer, J. (1993). Beyond assessment: Uruversitv/school collaboration in portfolio review and the challeng to program improvement. Detroit, MI: Wayne State Uruversity. (ERIC Document Reproduction Service No. ED 359 158)

Sparks-Langer, G. M., Simmons, J. M., Pasch, M., Colton, A., & Starko, A. (1990). Reflective pedagogical thinking: How can we promote it and measure it? Joumal of Teacher Education. 41(4). 23-32.

Stiggins, R. J., & Conklin, N. F. (1992). In teachers' hands: Investigatmg the practices of classroom assessment. Albany, NY: State Uruversity of New York Press.

Texas Education Agency. (1994). Texas School Duectory. Austhi, TX: Author.

Texas Education Agency. (1994, January). Leamer-centered school for Texas: A vision of Texas education. Austin, TX: Author.

137

Todd, R. F., & Reece, C. C. (1989, March). Desuable skills and knowledge outcomes for an introductory educational research course: A Delphi study. Paper presented at the Armual Meeting of the American Educational Research Association, San Francisco, CA.

Touzel, T. J. (1993, Febmary). Portfolio analysis: Windows of competence. Paper presented at the armual meeting of the American Association of Colleges for Teacher Education, San Diego, CA.

Uruversity Interscholastic League. (1996). UIL Constitution and Contest Rules. Austin, TX: Author.

Vacca, R. T., & Vacca, J. L. (1989). Content area reading. Glenview, IL: Scott, Foresman and Company.

Valencia, S. W., Hiebert, E. H., & Afflerbach, P. P. (Eds.) (1994). Authentic readmg assessment: Practices and possibilities. Newark, DE: Intemational Reading Association.

Van de Ven, A. H., & Delbecq, A. L. (1974). The effectiveness of nominal, delphi, and interacting group decision making processes. Academy of Management Joumal. 17(4). 605-621.

Webster's Ninth New Collegiate Dictionary. (1989). Springfield, MA: Merriam-Webster, Inc., Publishers.

Wiggms, G. (1992). Creatmg tests worth taking. Educational Leadership. 49(8). 26-33.

Wolf, D. P. (1989). Portfolio assessment: Sampling student work. Educational Leadership.46(7). 35-39.

Wolf, D. P., LeMahieu, P. G., & Eresh J. (1992). Good measure: Assessment as a tool for educational reform. Educational Leadership. 49(8). 8-13.

Wolf, K. (1991). The schoolteacher's portfolio: Issues in design, hnplementation, and evaluation. Phi Delta Kappan. 73(2). 129-136.

Worthen, B. R. (1993). Critical issues that wiU determine the future of altemative assessment. Phi Delta Kappan. 74(6). 444-454.

138

Zolhnan, A., & Jones, D. L. (1994, Febmary). Accomodating assessment and learning: Utilizing portfolios in teacher education with preservice teachers. Paper presented at the Annual Conference of the Research CouncU on Diagnostic and Prescriptive Mathematics (21st armual). Ft. Worth, TX.

139

APPENDIX A

SAMPLE LIST AND CORRESPONDENCE

140

TEXAS UNIVERSITIES REPRESENTED IN THE

SAMPLE OF HOME ECONOMICS

TEACHER EDUCATORS

A list of home econonucs teacher educators from the foUowmg state

uruvershies was provided by tiie Texas Education Agency, Home Econonucs Program

Division. All home econonucs teacher educators from each of the uruversities served

as the sample for this study.

AbUene Christian Uruvershy, AbUene

Baylor Uruversity, Waco

Lamar Uruversity, Beaumont

Prairie View State Uruversity, Prairie View

Sam Houston State Uruvershy, Huntsville

Southwest Texas State Uruversity, San Marcos

Stephen F. Austin State Uruversity, Nacogdoches

Tarleton State Uruversity, Stephenville

Texas A&M Uruversity - Kingsville, Kingsville

Texas Southem Uruversity, Houston

Texas Tech Uruversity, Lubbock

Texas Women's Uruvershy, Denton

Uruversity of Houston, Houston

141

U N I V E R S I T Y

Education, Nutrition and Resuurant/Hotel Management

Box 41162 Lubbock, TX 79409-1162 (806) 742-3068 FAX (806) 742-3042 May 16, 1995

Mr. Rick Howard Superintendent fralSD P.O. Box 140 Ira, TX 79527

Dear Mr. Howard:

In 1994, the Texas Education Agency adopted five proficiencies, outlined in the document Leamer-centered Schools for Texas: A Vision of Texas Educators, as the criteria to be used for preparing teachers in Texas. A portfolio is one suggested means of authentically assessing mastery of the proficiencies. (Institute for Performance-Based Accountability, 1994, draft).

You have been selected as one of 30 persormel administrators to participate in a statewide study to determine evidences that authentically assess the five teaching proficiencies outlined in Leamer-centered Schools for Texas. The study will be conducted in three rounds. The first two roimds will consist of checkUsts and will take approximately 30 minutes each. The final round will be sent to a random sample of the entire panel and will consist of reviewing and evaluating a portfolio prototype. The evaluation will take approximately one hour. You will have no expenses, other than time, coimected with the survey. All responses to the survey are confidential.

Your input will be used to develop a portfolio prototype that could be used to assess preservice home economics teachers. Directions are provided on the inside cover of the questionnaire booklet. Identification numbers on each instrument will be used for follow-up and data analysis purposes only.

Your response by May 31, 1995, will be appreciated. Your participation is critical and your opinions are valued. If you would like a copy of the results, please indicate on the inside front cover of the booklet. The booklet cover will be separated from the responses to maintain confidentiality.

Sincerely, , Sincerely,

^^^A-^ Lora Ann Neill, M.S. Betty L. Stout, Ph.D. Instructor/Teacher Educator Associate Professor Home Economics Education Home Economics Education Stephen F. Austin State University Texas Tech University

Enclosure

An EEO/Af/irmatwe Action Institution

142

U N I V E R S I T Y

Education, Nutrition and Restaurant/Hotel Management

Box 41162 Lubbock, TX 79409-1162 (806)742-3068 ^^ , ^ , « « , FAX (806) 742-3042 May 16, 1995

Dr. Marianna Rasco, Chair Department of Family and Consumer Sciences Abilene Christian University P.O. Box 8155, ACU Station AbUene, TX 79699-8155

Dear Marianna:

As a home economics teacher educator in Texas, you have been selected to participate in a statewide study to determine evidences that authentically assess the five teaching proficiencies outlined in Leamer-centered Schools for Texas: A Vision of Texas Educators (Texas Education Agency, 1994). Your help is needed for the success of this project. Your input will be used to develop a portfolio prototype to assess preservice home economics teachers.

The study will be conducted in three rounds. The first two rounds will consist of checklists and will take approximately 30 minutes each. The final roimd will be sent to a random sample of the entire panel and will consist of reviewing and evaluating a portfolio prototype. The evaluation will take approximately one hour. You will have no expenses, other than time, coimected with the survey. All responses to the survey are confidential.

Directions are provided on the inside cover of the questionnaire booklet. Identification numbers on each instrument will be used for follow-up and data analysis purposes only.

Your response by May 31, 1995, will be appreciated. Your participation is critical and your opinions are valued. If you would like a copy of the results, please indicate on the inside front cover of the booklet. The booklet cover will be separated from the responses to maintain confidenriality.

Sincerely, Sincerely,

Lora Ann Neill, M.S. Betty L. Stout, Ph.D. Instructor/Teacher Educator Associate Professor Department of Human Sciences College of Human Sciences Stephen F. Austin State University Texas Tech University

Enclosure

An EEO/AjJimuUive Action Institution

143

U N I V E R S I T Y


Box 41162 Lubbock, TX 79409-1162 (806)742-3068 ^ , , ^ , « « ^ FAX (806) 742 3042 May 16, 1995

Amy Jean Wolff 1511 Faro Dr. #121 Austin, TX 78741

Dear Amy:

In 1994, the Texas Education Agency adopted five proficiencies, outlined in the document Leamer-centered Schools for Texas: A Vision of Texas Educators, as the criteria to be used for preparing teachers in Texas. A portfolio is one suggested means of authentically assessing mastery of the proficiencies (Institute for Performance-Based Accoimtability, 1994, draft).

As a Spring 1995 home economics student teacher, you have been selected to participate in a statewide study to determine evidences that authentically assess the five teaching proficiencies outlined in Leamer-centered Schools for Texas. The study will be conducted in three rounds. The first two rounds will consist of checklists and will take approximately 30 minutes each. The final round will be sent to a random sample of the entire panel and will consist of reviewing and evaluating a portfolio prototype. The evaluation will take approximately one hour. You will have no expenses, other than time, coimected with the survey. All responses to the survey are confidential.

Your help is needed for the success of this project. Your input will be used to develop a portfolio prototype to assess preservice home economics teachers. Directions are provided on the inside cover of the survey booklet. Identification numbers on each instnmient will be used for follow-up and data analysis purposes only.

Your response by May 31, 1995, will be appreciated. Your participation is critical and your opinions are valued. If you would like a copy of the results, please indicate on the inside front cover of the survey booklet. The booklet cover will be separated from the responses to maintain confidentiality.

Sincerely,

fi^a^?)^ Lora Ann Neill, M.S. Betty L. Stout, Ph.D. Instructor/Teacher Educator Associate Professor Department of Human Sciences College of Human Sciences Stephen F. Austin State University Texas Tech University

Enclosure

An EEO/A/firmative Action Institution

144

U N 1 V E R S I T Y


Box 41162 Lubbock, TX 79409-1162 (806) 742-3068 FAX (806) 742-3042

June 7, 1995

Dr. Marianna Rasco, Chair Department of Family and Consumer Sciences Abilene Christian University P.O. Box 8155, ACU Station Abilene, TX 79699-8155

Dear Marianna:

Recently, I sent you a questionnaire asking you to check evidences that could authentically assess the five teaching proflciencies outlined in the document Leamer-centered Schools for Texas: A Vision of Texas Educators (1994). At this time, I have not received your response.

As a home economics teacher educator in Texas, your input is critical to the success of the study. Could you take a few minutes to complete the enclosed copy of the original questionnaiie? All responses to the survey are confidential. Identification numbers on each instrument will be used for follow-up and data analysis purposes only.

As you know, teacher preparation programs in Texas are already using the new proficiencies as a guide. Within the next year, a new teacher evaluation system will be developed based on the proficiencies. Your input on the questionnaire will be used to develop a portfoUo prototype that could be used to assess preservice home economics teachers. A portfolio is one suggested means of authentically assessing mastery of the proficiencies. (Institute for Performance-Based Accountability, 1994, draft).

Directions for completing the questionnaire are provided on the inside cover of the booklet Responses to the questionnaire will be reviewed, and a second questionnaire will be sent to you within a few weeks. Your input on both instiiiments is needed.

Your response by June 21,1995, will be appreciated. If you would like a copy of the results, please indicate on the inside front cover of the booklet. The booklet cover will be separated from the responses to maintain confidentiality.

Sincerely,

Lora Ann Neill, M.S. Instructor/Teacher Educator Department of Human Sciences Stephen F. Austin State University

Betty L. Stout, PhX). Associate Professor College of Human Sciences Texas Tech University

Enclosure

An EEO/AjJirmative Action Institution

145

Education, Nutrition and Restaurant'Hotel Management

Box 41162 Lubbock. TX 79409 1162 (806) 742-3068 FAX (806) 742 3042 September 5, 1995

Mr. Rick Howard Superintendent IralSD P.O. Box 140 ha, TX 79527

Dear Mr. Howard:

Thank you for responding to the roimd-one questionnaire seeking to determine evidences that authentically assess the five teaching proficiencies adopted by Texas Education Agency. Your prompt response to the brief round-two questionnaire will be appreciated. AU responses to the survey are confidential.

Please review each teaching proficiency and check ONLY those evidences that you feel will best assess the teaching proficiency described. Evidences that were selected by at least 70% of the roimd-one respondents are considered to have achieved consensus for this study and are deleted from the rotmd-two questionnaire. Evidences receiving less than 20% consensus in roimd one are also deleted from the rotmd-two questioimaire. The deleted responses for roimd two are shown in the left-hand boxes on the questioimaire.

When rotmd-two responses are received and simmiarized, a portfolio prototype will be developed that will include selections from those evidences that respondents deemed most authentic for assessing the teaching proficiencies. Selected respondents will be asked to review the completed portfolio to determine hs validity and usefulness. If you would be willing to review and evaluate the portfolio prototype, please complete the enclosed card. The review process should take no more than one hour of your time.

Your input on the round-two questionnaire is critical to the success of the project. Please take a few minutes to complete the quesionnaire and retiun your responses in the envelope provided by September 25. If you requested a copy of the study resuhs, a simmiary will be forwarded at the completion of the study. Thank you again for your participation.

Sincerely,

Lora Ann Neill, M.S.

Enclosures

An EEO/Affirmative Action Institution

146

DATE:


Lora Ann Neill

February 9, 1996

SUBJECT: Portfolio Prototype Contents

I am working on the contents of a portfolio prototype to be submitted to a group of educators and persoimel administrators for evaluation in assessing teaching proficiencies. The value of the portfolio prototype as an assessment tool for use with home economics preservice teachers will be enhanced by the inclusion of a variety of assessment strategies from teacher educators across the state.

The following list of potental portfolio items received the highest consensus in the two rounds of this study. If you have examples of any of the foUowmg items that you would be willing to share for possible inclusion in the portfolio prototype, please send them to me at the address below before March 1. Please complete and retum the enclosed release form along with your portfolio items.

Class Projects related to Teaching Field Creative Activity Instructional Materials (Teacher Designed) Leadership in Professional Associations Membership in Professional Associations Philosophy/Goals Statement Professional Development Activities (i.e.. Meetings Attended) Pupils' Evaluations Unit/Lesson Plans

In addition to the above, several of the second-round additions gamered strong consensus, and I am considering including some of these in order diat each teaching proficiency will have several items represented. If you have examples of any of the foUowing, I would iq>preciate your sharing them.

Evidence of Computer SkiUs or Use of a Variety of Telecommunications Evidence of Knowledge of Models of Teaching, Learning Styles, etc. Evidence of Attendance at ARD Meetings, Knowledge of lEPs, or Parent Contact

If your example is used in the portfolio prototype, your contribution to the prototype will be noted. I would appreciate any contribution you could make to the portfolio prototype. Send all examples and release forms to the address below.

Lora Ann Neill 204 E. 91st Street Odessa, TX 79765

Thank you for your participation.

141

RELEASE FORM

I give my permission for Lora Ann Neill to use the materials I have submitted

for possible inclusion in a portfolio prototype for assessing preservice home

economics teachers. I understand that student names and factors that

would be readily identifiable will be omitted from the examples and that the

format of the materials may be adjusted to fit the portfolio format. If my

work/materials are used, appropriate credit will be given in the final report.

Typed or printed name Permanent Address

Signature

Date Phone

RETURN TO


148

RESPONSE CARD FOR PORTFOLIO

PROTOTYPE REVIEW PANEL

TO: Lora Ann Neill

Yes, I will be willing to review the portfolio prototype and evaluate its validity and usability.

No, I am not willing to review the portfolio prototype.

I would like more information about the review process. Please call me at

Name

Current address

[This card may be returned with questionnaire or mailed separatelyj

149

TO: Home Economics Teacher Educators

FROM: Lora Ann Neill

DATE: September 5, 1995

SUBJECT: Portfolio Prototype Contents

Following a review of all responses to the roimd-two questionnaire, I will begin developing a portfolio prototype. The value of the portfolio prototype as an assessment tool for use with home economics preservice teachers will be enhanced by the inclusion of a variety of assessment strategies from teacher educators across the state.

If you would be willing to share examples of assessment strategies that would authentically assess any of the leamer-centered teaching proficiencies, please complete the release form included. Retum the form and any examples to the address below by October 1, 1995.

If your example is used in the portfolio prototype, your contribution to the prototype will be noted. If you would like to submit examples of your students' work, please copy the release form and obtain a completed release statement from each student. Retum the form, along with the smdent's work and a copy or simimary of the assignment that you gave.

Send all examples and release forms to


Thank you for your participation.

150

APPENDIX B

ROUND-ONE INSTRUMENT

151

EVIDENCES TO ASSESS TEACHING PROFICIENCIES

A study conducted by Lora Ann Neill, 126 Wortham Drive, Nacogdoches, TX 75961

152

GENERAL INSTRUCTIONS:

Section 1 of the questionnaire asks you to select and/or suggest evidences that would authentically assess the five learner-centered proficiencies developed for teachers in Texas.

"Authentic" assessment is defined in this study as any means of assessment that evaluates tasks, skills, or knowledge within a context of "real-life" situations, requires higher-order thinking, and/or allows the student to be an active participant in the assessment process.

Section 2 is a brief general information section. Your input is needed for both sections.

A second round of questions, based on collected responses to this first round, is planned to follow in approximately three weeks.

NOTE: If your address has changed, please give your summer address. This sheet will be separated from your responses.

Name

Address

Phone

would like to receive a copy of the study results.

153

DIRECTIONS: Please read the proficiency statement in the left-hand column. CHECK ( / ) the evidence(s) in the box on the right that you believe would BEST ASSESS the teaching proficiency described. Any other evidence(s) that could BETTER assess the proficiency can be added in the spaces provided.

TEACHING PROFICIENCY 1: Learner-centered Knowledge


{Leamer-centered Schoots for Texas: A Vision of Texas Educators. Texas Education Agency, 1994, p. 3)

EVIDENCES OF TEACHING PROFICIENCY 1


Class Prorects Related to Teaching Field

Check (/>


Creative Activity



Oescnption of Teaching Strategy Used

Documentation of Field Expenences Pnor to Student Teaching



ExCET Scores


Instnjctional Materials (Teacfier Designed)



Membership in Professional Associatioru



Professional Development Activities He., Meetings Attended)

Public Relations Protect

Pupils' Evaluations

Refleaive Journal of Student Teacher

Research Project

Resume


Self-assessment Checklist/essay

Student Teaching Evaluations (from Supervising Teacher, University Supervisor, Etc.)

Transcnpt

Unit/Lesson Plans


OTHER (Please Spmafyl

154

DIRECTIONS; Please read the proficiency statement in the left-hand column. CHECK ( / ) the evidence(s) in the box on the right that you believe would BEST ASSESS the teaching proficiency described. Any other evidence(s) that could BETTER assess the proficiency can be added in the spaces provided.

TEACHING PROFICIENCY 2: Learner-centered Instruction

To create a leamer-centered community, the teacher collaboratively identifies needs; and plans, implements, and assesses instruction using technology and other resources.

{Leamer-centered Sct)ools for Texas: A Vision of Texas Educators, Texas Education Agency, 1994. p. 4)



Class Proiects Related to Teaching Field


Creative Activity



Description of Teaching Strategy Used

Documentation of Field Experiences Pnor to Student Teaching



ExCET Scores








Professional Development Activities (I.e., Meetings Attended)


Pupils' Evaluations

Reflective Joumal of Student Teacher

Research Project

Resume



Student Teaching Evaluations (from Supervising Teacher, University Supervisor. Etc.)

Transcript

Unit/Lesson Plans


1 OTHER (Please Sfttity)

1

Check (•)

155

DIRECTIONS: Please read the proficiency statement in the left-hand column. CHECK ( • ) the evidence(s) in the box on the right that you believe would BEST ASSESS the teaching proficiency described. Any other evidencels) that could BETTER assess the proficiency can be added in the spaces provided.

TEACHING PROFICIENCY 3: Equity in Excellence for All Learners

The teacher responds appropriately to diverse groups of learners.

[Leamer-centered Schools for Texas: A Vision of Texas Educators. Texas Education Agency, 1994, p. 5)





Creative Activity







ExCET Scores


Instnjctional Materials (Teacher Designed)






Professiorul Development Activities (I.e.. Meetings Attended)


Pupils' Evaluations

Reflective Joumal of Student Teacfier

Research Project

Resume



R Studem Teaching Evaluations (from Supervising R Teacher, Unlversrty Supervisor, Etc.)

Trarucript

Unit/Lesson Plans


OTHER (Please Spwdfy)

Check (/)

156

DIRECTIONS: Please read the proficiency statement in the left-hand column, CHECK ( / ) the evidencels) in the box on the right that you believe would BEST ASSESS the teaching proficiency described. Any other evidencels) that could BETTER assess the proficiency can be added in the spaces provided.

TEACHING PROFICIENCY 4: Leamer-centered Communication

VJhile acting as an advocate for all students and the school, the teacher demonstrates effective professional and interpersonal communication skills.

[Leamer-centered Scliools for Texas: A Vision of Texas Educators, Texas Education Agency, 1994, p. 6)





Creative Activity







ExCET Scores





Membership In Professional Associations





Pupils' Evaluations


Research Project

Resume




Transcnpt

Unit/Lesson Plans


OTHER (Please Spectfyl

Check (/)

1

157

DIRECTIONS: Please read the proficiency statement in the left-hand column. CHECK (• ) the evidencels) in the box on the right that you believe would BEST ASSESS the teaching proficiency described. Any other evidencels) that could BETTER assess the proficiency can be added in the spaces provided.

TEACHING PROFICIENCY 5: Learner-centered Professional Development


[Leamer-centered Scfiools for Texas: A Vision of Texas Educators. Texas Education Agency, 1994, p. 7)





Creative Activity







ExCET Scores










Pupils' Evaluations

Reflective Joumal of Student Teacher

Research Project

Resume




Transcript

Unit/Lesson Plans


OTHER (Please Spedfy)

Check (•)

158

HOME ECONOMICS TEACHER EDUCATORS

GENERAL INFORMATION - Please respond to the following questions.

1 • CHECK (•) the response corresponding to the number of years you have taught at the college level.

a. 0 - 5 years

b. 6 - 1 0 years

c. 11-15 years

d. 16-21 years

e. 21 -I- years

2. CHECK (•) the response corresponding to the number of years of public school teaching experience you have.

a.

b.

c.

d.

e.

f.

0 years

1 - 5 years

6 - 1 0 years

1 1 - 1 5 years

16 - 20 years

21 -1- years

CHECK (•) the number of student teachers you have directfy supervised in field-based experiences since August 31, 1993.

a.

b.

c.

d.

e.

f.

g-

h.

i.

0

1 - 4

5 - 8

9 - 12

13- 16

1 7 - 2 0

21 -24

2 5 - 2 8

29 or more

4. CHECK (/) the way(s) you currently use portfolios.

a. I require students to have a portfolio in one or more courses.

b. I require students to have a portfolio during student teaching.

c. I currently do not use portfolios in my classes.

d. Other (please specify):

RANK the following purposes of portfolios to your teacher education program from 1 - 3, using 1 as most imnnrtiint anri "i ac lonct imnnrtitnt important and 3 as least important.

a. shows mastery of content

b. shows proficiency in teaching

c. markets competencies and skills

159

What assessment strategies from your home economics education courses could be used in a portfolio? (Please list examples)

THANK YOU FOR YOUR RESPONSES!

Return the questionnaire booklet in the stamped and addressed mailer provided. Expect a second round of questions within approximately three weeks!

160

HOME ECONOMICS PRESERVICE TEACHERS

GENERAL INFORMATION - Please respond to the fo l lowing quest ions.

1. CHECK (/) the response that best describes the type of teacher certification program in which you were/are enrolled.

a. Undergraduate teacher education program

b. Post-baccaiaureate program

c. Jamison Bill program

When will / did you graduate?

month year

3. CHECK (•) all responses that apply to you.

a. Still in school.

b. Currently not seeking employment.

c. Employed for fall as a home economics teacher.

d. Looking for a home economics teaching job in Texas.

e. Looking for a home economics teaching job other than Texas.

f. Looking for a home economics Extension position.

g. Looking for a job in another employment area.

Please specify the employment area:

CHECK (•) your experience with portfolios. ( Check (/) all that apply.)

a. I have never developed a professional portfolio.

b. I developed a portfolio as a requirement in a course.

c. I developed a portfolio during student teaching.

d. I have used a portfolio in an interview for a teaching position.

e. I have used a portfolio in an interview for a non-teaching position.

Please specify the non-teaching position:

RANK the following reasons for developing a portfolio, using 1 as most important and 5 as least important

a. to demonstrate personal development

b. to market myself

c. to demonstrate proficiency in teaching

d. to draw closure on my teaching preparation

e. to demonstrate knowledge about a subject

161

What evidences of teaching or assessment strategies from your professional development courses or from your teaching experience could be used in a portfolio? (Please list specific examples.)


Return the questionnaire booklet in the stamped and addressed mailer provided. Expect a second round of questions within approximately three weeks! Make sure you have provided your summer address on the inside front cover.

162

PERSONNEL ADMINISTRATORS

GENERAL INFORMATION - Please respond to the following questions.

1. What is your current position title?

2. CHECK (/) the response corresponding to the number of years you have held your current position,

a. 0 - 5 years

b.

c.

d.

e.

CHECK { / ) the

a.

b.

c.

d.

e.

6 - 1 0 years

1 1 - 1 5 years

1 6 - 2 1 years

21 -1- years

i size of your school district

A

AA

AAA

AAAA

AAAAA

4. CIRCLE the number of the Education Service Center region that serves your school district.

1 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20

5. CHECK (/) your gender.

a. female

b. male

6. CHECK (•) your level of education.

a. bachelor's degree

b. master's degree

c. doctorate

d. Other (please specify)

CHECK (•) the response corresponding to the number of years oi public scAoo/teaching experience you have.

a.

b.

c.

d.

e.

f.

0 years

1 - 5 years

6 - 1 0 years

1 1 - 1 5 years

16 - 20 years

21 -t- years

163

8. CHECK ( / ) the response corresponding to the number of years of public school administrative experience you have.

a. 0 years

b. 1 - 5 years

c. 6 - 1 0 years

d. 11 15 years

e. 1 6 - 2 0 years

f. 21-1- years

9. CHECK ( / ) the number of prospective teachers you have interviewed in the past school year.

a. 0

b. 1 - 1 5

c. 1 6 - 3 0

d. 31 - 50

e. 51 - 7 5

f. 76 or more

10. RANK the items below from 1 - 6 to indicate the value you place on each in the hiring process, using 1 as most compelling and 6 as least compelling.

a. transcript

b. resume

c. letters of recommendation

d. teaching evaluations

e. written philosophy

f. sample unit/lesson plans

11. CHECK ( / ) the number of prospective teachers who have submitted portfolios in the past year to illustrate their teaching proficiency.

a.

b.

c.

d.

e.

f.

none (If you answered none, skip to question 15.)

1 - 5

6 - 10

1 6 - 2 0

21 -25

more than 25

12. CHECK ( / ) the average amount of time you spend reviewing each prospective teacher's portfolio.

a. 1 - 5 minutes

b. 6 - 1 0 minutes

c. 1 1 - 1 5 minutes

d. 1 6 - 2 0 minutes

e. more than 20 minutes

164

13. CHECK ( / ) when you review portfolios. (Check ( / ) all that apply.)

a. Before the interview

b. During the interview

c. After the interview

d. In lieu of an interview

e. Other (please specify)_

14. CHECK ( / ) how you review the portfolio. (Check ( / ) all that apply.)

a. Alone

b. With colleagues

c. With the interviewee

d. Other (please specify)

15. To be answered only if you selected none to question 11 above.

If you were to review a prospective teacher's portfolio, how much time do you feel you could spend reviewing it?

a.

b.

c.

d.

e.

1 - 5 minutes

6 - 1 0 minutes

1 1 - 1 5 minutes

16 - 20 minutes

more than 20 minutes


Return the questionnaire booklet in the stamped and addressed mailer provided. Expect a second round of questions within approximately three weeks!

165

APPENDDC C

ROUND-TWO INSTRUMENT

166

EVIDENCES TO ASSESS TEACHING PROFICIENCIES

Round 2

A study conducted by Lora Ann Neill, 204 E. 91st Street, Odessa, TX 79765.

167

DIRECTIONS: Please read the proficiency statement in the left-hand column. Check ( / ) ONLY the

evidencels) in the box on the right that you believe would BEST A S S E S S Teaching Proficiency 1.

Evidences that received 70% or greater agreement or less than 20% agreement in round one have been deleted from the round-two questionnaire and are listed in the boxes at left below. Additional evidences suggested by round-one respondents are noted with an asterisk (*).

TEACHING PROFICIENCY 1: Leamer-centered Knowledge


[Leamer-centered Scfiools for Texas: A Vision of Texas Educators, Texas Education Agency, 1994, p. 3)

/• romd ta*. tt* ftllmntf tndtaci af Ttaetitf Profiettacf I mtitd 70% agrfmtat nd it dtlMtd Inm romd twK

Imtnictitnal Uttmitls (TMCIW Ofsigntdl

/( rtaad m; tkt follmnti tndtaets tf Tnekiii frtfidiacf I r—ettd Ins Itn 21% a§rtimtt Mtf art Mittd from rwmi twt:

Evidtnct ol Community Strvict Entnprtnturship PItn Public Rilitions Pnjtct


* Ability to Use a Variety of Telecommunications



Classroom Procedures/Oiscipline Plan

Creative Activity


Oescnption of Teaching Strategy Used

Documentation of Field Expenences Pnor to Student Teaching


• Evidence of Use of Appropriate Software

ExCET Scores




fVlembership In Professional Associations

* IWentorships




Pupils' Evaluations

* Pupil Projects as a Result of Lesson


Research Project

Resume




Transcript

Unit/Lesson Plans

VIdeo/Audiotape of Lesson

Check (/)

168

DIRECTIONS: Please read the proficiency statement in the left-hand column. Check ( / ) ONLY the

evidencels) In the box on the right that you believe would BEST A S S E S S Teaching Proficiency 1

Evidences that received 70% or greater agreement or less than 20% agreement in round one have been

deleted from the round-two questionnaire and are listed in the boxes at left below. Additional evidences

suggested by round-one respondents are noted with an asterisk (*).

TEACHING PROFICIENCY 2: Learner-centered Instruction

To create a learner-centered community, the teacher collaboratively identifies needs; and plans, implements, and assesses instruction using technology and other resources.

[Leamer-centered Schools for Texas: A Vision of Texas Educators, Texas Education Agency, 1994, p. 4)

M r—ad mt, ttt ftHmtiag trUtacu tf Ttattiat Fnfkifcr 2 r—ettd 70% a§rttmtat aad tn dthltd I

' twi:

Otti Pmjicts Htltttd to Tttdmg fttld Crttthn Activity tatructienil Mtlitiilt (Tttchtr Ditigntdl UnlA-tsson Pttm

la rtad tat, tkt fiUmtmi tridtaett af Ttaetiti Pnfidtacr 2 rtaettd htt tktt 20% a/rttmtat aad art dtltttd frtm rtmad two:

Adttvimtnt Annnls/Ctrtificttis Entrtprmmnhip Pttn ExCET Scans Lttdirsltp it ProftssiontI Assodttions Litttrs of Htcommtndttian Mtmbtrstiip in Frolissiantl Associttions ntsunt Transcript








* Evidence of Collaboration on Projects

* Evidence of Knowledge of Models of Teaching, Learning Styles, etc.



Photograpfis of Teaching Activities



Pupils' Evaluations


Research Project




VIdeo/Audiotape of Lesson

Check ( • )

169

DIRECTIONS: Please read the proficiency statement in the left-hand column. Check (/) ONLY the evidencels) in the box on the right that you believe would BEST ASSESS Teaching Proficiency 1.

Evidences that received 70% or greater agreement or less than 20% agreement in round one have been deleted from the round-two questionnaire and are listed in the boxes at left below. Additional evidences suggested by round-one respondents are noted with an asterisk (*|.

TEACHING PROFICIENCY 3: Equity in Excellence for All Learners

The teacher responds appropriately to diverse groups of learners.

[Leamer-centered ScfHX>ls for Texas: A Vision of Texas Educators. Texas Education Agency, 1994, p. 5)

la raaad aat, tkt ftUamaf tndtatts af Ttaclaa§ Prtfidmtf 3 rtaektd 70% agnamtat tad an dtltttd frtm raaad twa:

diss Pnjtcts Rtltttd to Tttdanff RiU ktsoueticntl Mttiriils (Tttchtr Dtsigntdl Pupils' Enlatoons UnitA-isson Plans

It raaad tat, ttt IMaariai thdtaett af TiatUai Pnfjdtaer 3 rtacitd last tkaa 20% t/rttmtat aad in dtltttd frtm raaad two:

Achitvtatnt AwtrtlsXinificiles EntnprtrmrsHp PItn Evidtnct of Computtr SkWs ExCET Scons Lttdtrsiiip in Proltssional Associttions Litttrs of Hicommwdttion Mimbtrship in Pnltssionil Assoditicns Proftssionil Ot¥t/apmtnt Activitin f.t.. Millings Attmdtdl Piddic HtUtions Prcfict Risitrch Pro/icl Risumi Transcript




Creative Activity





* Evidence of Attendance at ARD Meetings or Knowledge of lEPs

* Evidence of Outside School Activities During Student Teaching


Instructional Materials (Teacfier Designed)







Unit/Lesson Plans


Check (/)

170

DIRECTIONS: Please read the proficiency statement in the left-hand column. Check (/) ONLY the evidencels) in the box on the right that you believe would BEST ASSESS Teaching Proficiency 1.


TEACHING PROFICIENCY 4: Learner-centered Communication


[Leamer-centered Schools for Texas: A Vision of Texas Educators. Texas Education Agency, 1994, p. 6)

la raaad aat, at tiidtacaa af Ttactia/ fraSdaacf 4 nactid 70% a/ntmtaL

la raaad mil. tit ItHamag twidtaen af Taaitrng PnSdiaer 4 naetad lau tkaa 20% t/nmatat aad i dtltttd fnm raaad tan:

bitrtprmmrs/ip PItn ExCET Scons Ptiotognphs of Tucking Activitits Rtsun/i Pra/ict Stnfilts af Pupis'Wort Transcript

EVDBiCES OF TEACHMG PROFICIBiCY 4




Creative Activity



* Evidence of Parental Contact in Student Teaching

• Evidence of Teaching Experience in Volunteer Efforts (i.e., church, community agency)






Leadersfiip in Professional Associations


Membership In Professior\al Associations




F>upils' Evaluations


Resume



Unit/Lesson Plans


CtMCtll/)

171

DIRECTIONS: Please read the proficiency statement in the left-hand column. Check (•) ONLY the

evidence(s) in the box on the right that you believe would BEST ASSESS Teaching Proficiency 1.


TEACHING PROFICIENCY 5: Learner-centered Professional Development


[Learrter-centered Schools for Texas: A Vision of Texas Educators, Texas Education Agency, 1994, p. 7)

la raaad aat, ttt fatamag PitCiimt) 5 nttktd 70%

af Ttaekiag tat aad an dtlmd I

Lttdmstip in Profuiiaitl Associttions Uttnturs/tp in Proinsionil Associttions nOotaplir/Botls Stitimtnl Profusiantl Dmlopmint Activitiis fj, UiUjngs Attmtlidl

la raaad tat, tkt ftUtmag tridtatts af Ttaciag Prafidtacr 5 naektd lass tkaa 20% t/ntmut aad in dtltttd fnm naad two:

Class Projacts Rtlittd to Tucking HtU Dataiptim of Tucking Stntigy Usid Daeunantition of Hold bpirinKis Prier to Studmt Tucking ExCET Scons Phatagnphs of Tiacking Activitits Pupils' [vtkjivens Siofdts of Pupis'Work UnitAisson Plms VOiolAudiotapt al Ltsscn

EVK>«CES OF TEACHMG PftOFIOBiCY 5


* Case Study of Ethical Dilemma


Creative Activity

1 Evidence of Communrty Service

1 Evidence of Computer Skills

1 Entrepreneurship Plan

Evaluation Devices (Teacfier Designed)


Irutructional Materials (Teacher Designed)



Reflective Journal of Student Teacfier

Research Project

Resume

Self-assessment Cfiecklist/essay

Student Teaching Evaluations (from Supervisir>g Teacfier, University Supervisor. Etc.)

1 Transcript

CiMClcl/)

Thank you for your participation in this study.

Ill

APPENDIX D

SELECTIONS FROM THE PORTFOLIO PROTOTYPE

173

EXAMPLE OF A REFLECTIVE STATEMENT

Evaluations from High School Students

During my student teaching, the evaluations from both my supervising teacher and my university supervisor were helpftil in alerting me to problem areas in my teaching and in encouraging me to continue in areas where I was doing well. Students, too, provided information about my teaching performance, sometimes vividly or jokingly:

• • • "Hey, Miss, this is boring." • • • "Can we do that again tomorrow?" • • • "Why we gotta learn this stuff for?" • • • "Miss, you never told us about that."

I am glad that my supervising teachers let my students provide evaluations which give me concrete evidence on which to base future teaching goals and strategies.

Since the evaluations were anonymous, my examples may not represent each ethnic group in the classroom; they do represent, however, both genders. The conunent on the third evaluation revealed the identity of the respondent.

This particular young man was constantly wearing his cap in the hall and into class, which is against school and classroom rules. I tried discipline strategies from ignoring to referral, but the cap and his behavior was a constant source of discord between the two of us. While his evaluation represents one of the most negative ones that I received, I was surprised that he gave me so many positive checks.

In retrospect, in order to reinforce appropriate behavior, I should have given him additional positive attention on days he did not bring the cap to class. In future classroom situations, I might provide a hat/cap rack so that students can receive recognition for "Hangin' with Ms. Blair."

174

'a ,H

PROFESSIONAL PORTFOLIO

Tracy Helen Blair

#34 Trask Lane Fairview, Texas 78421

(419) 687-2234

i ^

175

m

IS II

GOAL STATEMENT

The purpose of this professional portfolio is to present my knowledge, skills, and experiences in a manner that will enable me to secure a teaching position in home economics. The evidences included in the portfolio attempt to show my competencies in the five leamer-centered teaching proficiencies adopted by the Texas State Board of Education.

Tracy Blair

a 5L 11

=1 176

w ORGANIZATION OF PORTFOLIO

The professional portfolio is divided into three major sections: • Background Information, • Information from Others, and • Products of Teaching.

Each section is separated into subdivisions as outlined in the Table of Contents. Each section begins with a reflective statement that describes the items in the subdivisions and the rationale for including each item of evidence.

The evidences presented in the portfolio are aligned with the teaching proficiencies outlined in Learner-Centered Schools for Texas; A Vision of Texa.s Educators (Texas Education Agency, 1994) in the following manner:

• Leamer-centered Knowledge Introduction

Rauonale for Teaching as a Career Career Goals Philosophy of Teaching Home Economics

University Class Projects Related to Teaching Field Technology Projects Professional Acdviues Lesson Plans

• Leamer-centered Instmction University Class Projects Related to Teaching Field Evaluauons from High School Smdents Block Plan Lesson Plans Teacher-developed Materials Technology Projects Creative Activities Special Needs Students

• Equity in Excellence for All Leamers Evaluations from High School Students Special Needs Smdents 4MAT Lesson Plan Technology Projects

• Leamer-centered Communication Parent Contacts Technology Projects Letters from High School Smdents Block/Lesson Plans

• Leamer-centered Professional Development Professional Activities Honors and Awards Certifications Career Goals

111

rfll

177

Table of Contents

Goal Statement ii

Organization of the Portfolio ill

Table of Contents iv

Background Information Background Information 6 Introduction 9

Professional Activities/Organizations 14

Honors and Awards 19

Certifications 24

University Class Projects Related to Teaching Field 28

Information from Others Information from Others 38

Evaluations from High School Students 40

Letters from High School Students 45

Products of Teaching Products of Teaching 50

Unit/Lesson Plans 52

Teacher-Developed Materials 66

Creative Activities 80

Technology Activities 90

Samples of Students' Work 100

Parent Contacts 107

Special Needs Students 113

Summary Statement 123

IV

178

^ iL-

BACKGROUND INFORMATION

' ^ ^^

179

Background Information

The Background Information section includes the following subdivisions. Introduction Professional Activities Honors and Awards Certifications University Class Projects Related to Teaching Field

Introduction

The Introduction subdivision provides evidences of Leamer-centered Knowledge through information about my reasons for choosing teaching as a career and highlights of two memorable learning experiences that have fueled my resolve to teach. An explanation of my path to choosing home economics as a teaching field is provided to portray my deep commitment to my career (Leamer-centered Professional Development).

Separate discussions of my student teaching experience, career goals, and personal philosophy of teaching complete the first part of this section. I feel that these are important to include as evidences of Leamer-centered Knowledge, Leamer-centered Instruction, and Leamer-centered Professional Development so that the reader can get a clearer picture of my attitudes and values. My attitudes and values provide a foundation for my skills and talents in teaching.


I have included a brief description of each of the professional groups of which I am, or have been, a member. A timeline of my professional development activities reveals how I have grown and developed as a professional. Tlie timeline shows my conunitment to maintaining my professionalism (Leamer-centered Professional Development).

Honors and Awards

During my college career, I received several honors and awards. I have included a selection of these in order to show to the reader how I strive toward excellence in everything that I do. Except for academic honors, honors and awards have not been my goals; rather, they have been an encouragement to me to continue to become the best that I can be (Leamer-centered Professional Development).

6

^ ,^^,, li

180

°a [t

Background Information, continued

Certifications

Certifications in HyperStudio and 4MAT Fundamental Training are included because I use these skills both personally and professionally. With HyperStudio I prepared computer transparencies that accompany my lesson presentations, and students used the software to create computer reviews in HDF and PCD classes. 4MAT lessons are structured to reach each student's individual learning style so that all students in the class leara in their "best" style. This subdivision offers evidences of mastery in Leamer-centered Instmction, Equity in Excellence for all Leamers, and Leamer-centered Professional Development.

University Class Projects Related to Teaching

Several projects that I developed in university classes proved helpful as I planned for classes during my student teaching. Two such projects are included as evidences of Leamer-centered Knowledge and Leamer-centered Instmction.

i 181

H U

o

>

Q

o

O P PN

*s es

u a

• <

=3

'1 • = 5 ^ ka . S QA M si

^ 2 o -o o = (- O W O

2> o

E 2 g •« u s s S

Si

= .. 1 ^ 2 •3 8 ^ 8

S C U 3 e V > o o = o g

O < Z 5

H

X

l 5

^ 3

u .S

H Si-

a a E < ft

E VI

1. S >.

^ 3 3

J« | S 1 u S u 2(^ 3

i« £ n >. M 8 3 «fl

£• ° I .S -s

f "

<

= .= ^

< 3 a£

5 "

III Z a£ U]

^ • 1

j l S at S: 2.

u ( S u c

S « S

- a" • .5 H «» s ^ 3 3 E S a£ &

8

S < M £ a

£ "S •=

s i s

f i 1811 U £ < E ec §•

es o o U U U

r

3

U <£ Q

s I I

Si

182

B or fi

Information from Others

The Information from Others section contains two subdivisions: • Evaluations from High School Students and • Letters from High School Students.

The week before I completed my student teaching, Ms. Worthen and Mrs. Chaney asked students in each class I taught to complete an evaluation sheet on my teaching performance. The activity was voluntary and anonymous, and responses were completed and collected while I was "out of the room." Five students present in classes that day did not submit evaluation sheets.

The evaluation responses were typical to the three included in this section. Most were overwhelmingly generous in their praise; but I found that the critical comments were very similar to the comments made openly in class. This surprised me because I thought the anonymity of the checklist would gamer more negative conunents.

From the evaluations, I leamed that most students viewed my teaching performance favorably and that I can depend, for the most part, on their vocal comments in class to reveal how they feel about the class and my teaching. During student teaching, I often took their vocal comments too personally. Ms. Worthen and Mrs. Chaney helped me use students' comments to improve my lessons and my responses to pupils' needs.

My supervising teachers encouraged students to write me "farewell letters" to present to me on my final day with them. The letters were given to me by pupils in my classes and by FHA members who were not in my classes. I included three letters which are typical of the ones I received. I value the opportunity I had to help students take small steps toward becoming responsible adults, and I treasure their letters of ^preciation.

Evaluation

Students feel that I spend too much time talking and that I rush them through activities. My nervousness as a new teacher may have contributed to this, but I need to be aware of these characteristics as I begin my teaching career. Students enjoy expressing themselves; I need to talk less and listen more. Students often need more time to complete projects than I have allocated; lessons can be extended to the next day or revised to fit the time. Leam to be flexible.

(The evaluations and letters are provided as evidence of Leamer-centered Instmction, Leamer-centered Conununication, and Equity in Excellence.)

38

^ ^n

184

STLDE.NT TE.ACHER EV.ALL.ATIG.N

R-ATIN'G F.-\CTORS 1 E.XCELLENT 1

Does the teacher s eem^U-preoared for lecture and labs

Does the teacher shovw conridence before the class

Does the teacher allow enough time for completion

of activities Does the teacher make you

feel comfonabie to ask question

Does the teacher show genuine interest in teaching

the course Does the teacher

successfully use discipline Does the teacher

communicate well with students

Does the teacher relate positively and fairly to all

students Does the teacher exhibit a

professional charaaeristic in appearance, language,

responsibilirv. dependability

y y

/

/

J

V

J

V

ABO\T AV-ER.\GE

/

.AVER-^GE BELOW AV-ER.A.GE

L'N'SATIS-F A C T O R Y

Please answer these questions honestly.

Please suggest ways in which the teacher can improve his/her teaching.

What did you like least about his/her way of teaching?

What did you like most about his/her way of teaching?

41

185

3S HI

Products of Teaching

The Products of Teaching section includes the following subdivisions: • Unit and Lesson Plans, • Teacher-Developed Materials, • Creative Activities • Technology Activities, • Special Needs Students, • Samples of Students' Work, and • Parent Contacts.

My student teaching experience has been the most intense leaming period of my life. I entered student teaching with some apprehension and much excitement; but, also, with the certainty that I knew almost everything there was to know about teaching. After all, I am the child of a teacher; I have been a blossoming teacher from the begiiming; and my work and school experiences have given me a wealth of practice in preparing for teaching.

The first week at Hartley High School found me supervising detention hall, directing the production of a play for the Junior Assembly, and tying 300 ribbons around Valentine Socks for Your Sweetie. They never taught us these skills in methods classes!

In between activities that first week, I observed home economics classes taught by my supervising teacher, Ms. Angela Worthen, and the other two home economics teachers, Ms. Toni Chaney and Mrs. Hannah Mears. I was inunediately involved with the students, however, and loving every minute of the school day (and up to four hours after school and at night).

I have included in this section a selection of the materials that I developed or utilized during my student teaching experience. Each subdivision begins with an explanatory/reflective statement about the materials included. The evidences in the Products of Teaching section attempt to show my competencies in four of the five proficiencies: Leamer-centered Knowledge, Leamer-centered Instmction, Equity in Excellence, and Leamer-centered Communication.

50

i 187

Unit Plans and Lesson Plans

Ms. Worthen teaches two sections of Home Economics Cooperative Education (HECE) and a Parenting and Child Development class. She supervises students on the job during 6th and 7th periods. Her 4th period time slot is for planning and for supervising Lunch Detention Hall. In order for me to get a broader view of the home economics program and to have more experiences in classroom teaching, she suggested that I work with Mrs. Toni Chaney, who teaches Housing, Design, and Furnishings (HDF), during 5th period. My daily schedule during smdent teaching was as follows:

1st period Home Economics Cooperative Education (HECE) I 2nd period HECE II 3rd period Parenting and Child Development (PCD) 4th/5th period Housing, Design and Fumishings (HDF) (Mrs. Chaney) 6th period Planning period/Supervision of HECE students 7th period Supervision of HECE smdents

UNITS/LESSONS TAUGHT

(I assumed total responsibility for 91 class periods listed below. I assisted Ms. Worthen and Mrs. Chaney in teaching approximately 72 other classes by supervising individual and group work, presenting mini-lessons, and answering smdents' questions.)

HECE I Money Management - 3 weeks (9 class periods) Careers in Child Development - 3 class periods On-job Supervision - 25 class periods

HECE II Writing a Resume - 6 class periods (over a 4 week period) Entrepreneurship - 2 weeks (10 class periods)

PCD

HDF

Substimte Care - 2 class periods Observing Children - 2 class periods The One-year-old Child - 2Vi weeks (11 class periods) Selecting Toys, Books, and Games - 4 class periods

Housing styles - 1 week (4 class periods) Design elements - 3 weeks (15 class periods)

I have included a block plan from my HECE II Entrepreneurship unit. Block plans serve as an outline for my teaching. During my smdent teaching, I learned that the more completely I can plan a unit on a block plan form, the easier I can adjust my teaching plan when changes occur in the school activity calendar. While 1 noticed that

52

188

Unit Plans and Lesson Plans, continued

experienced teachers use block plans as their only lesson plans, I still need the security of having complete lessons planned, even if I end up combining lessons.

A variety of lesson plans are included in the portfolio. All lesson plans, except for the last two, follow the Lesson Cycle. I tried to include plans that represent a variety of leaming strategies and teaching models, including. Inductive Thinking, Direct Instmction, Simulation, and Synectics.

During my smdent teaching, I was privileged to attend a 4MAT Fundamental Training session. Hartley High School faculty members conducted the 3-day training, and the school paid for the instmction guides and materials we were given. Substimtes were provided so that classroom teachers could attend. The 4MAT system is a way of planning to address all leaming styles of smdents. A balance of left-brain and right-brain activities are provided.

During the training, I adapted a lesson that I had taught on Line ftom the Design Elements unit into the 4MAT plan. The 4MAT lesson plan will probably be more enjoyable for all smdents than the original lesson I taught.

Later when I taught the mini-unit on Selecting Toys, Books, and Games, Ms. Worthen suggested that I develop the lesson on Books as a 4MAT lesson. The smdents and I had so much fun with this lesson that we extended the unit by one day. 4MAT varies activities in a lesson and attempts to provide oppormnities for all leaming styles to excel. It is easy to use the very best of the Lesson Cycle and 4MAT to provide exciting and worthwhile lessons. The last two lessons in this subdivision represent 4MAT plans.

The items in this subdivision attempt to show my compentencies in Leamer-centered Knowledge and Leamer-centered Instmction. Equity in Excellence is shown in the 4MAT lesson plans and in the Unit Plan. Leamer-centered Communication is addressed in the Unit Plan as I invited parents and community leaders into the classroom.

189

<

0)

c

a. 0)

< o cc

X

<

Q.

O m

U)

LU

< O

o U J

ro .c $

</) V) <l> r (fl 3

ffi <TJ

O

O )

c c

Ian

U .

o SI

(A

c lA 3

0 0

n

. o

log

a> E 03

z r..

« u v 5

hal

(A

(A rk

et

to

k o> c w 0)

(0

c (D

log

(A

c _ (0 =

| f i 0}

<A « « (0 V (O c o

3 —

(0 c

w 9 O u>

a a. .<£

> < a CO UJ 3 »-

(A V

(A 0 )

c

Bus

. o v> 0 ) Q .

>•

c3

UJ

U liJ X

D O

u

O z o

(O

.c

a. (A

0 ) 3 C

Q . 0)

i n

.o

U

u o z t -

01

anc

por

E 0)

£

* •

n) c CO

$ - J

(—

1/1 <A 0)

c (A 3

S3

C

C

o ^-^ c O l o u

0)

F CO

c

• D c 10

11 " ^ S (A

E « CD E C <A

_ 3 CO . Q C

•— " O

gi 5 0 o c °-m 2 £ " 1 ° o C

> CD _ l o H "iA

0) . ^

C » CD E O 03

E'i o> c *.. 4... 0) (A

0> U

O CD

Q .

$ ™ $

O )

<A .SJ <A —

.E 0)

<A >

CO

(D 0 > (A

'S 9} (A w

i I

.c CA

^ 3 9) C

« ^ a. 9)

r V

ine

0) • o

$ _ J

1-

, 3 01

c 9)

epr

* r f

c «

T 3 C CD

O OQ -J UJ U I -

ca CE O < O

0 —

01 w

^ E C 3 CD (A

£ "> C ™ «1 CD O ) 0) _:

o = - o (A (A CD 2 ttJ J3 ^ _ 3 E -o "5 CO CD 4) »— > C (A . , OJ — O 03

£ 2 Q . -

- o o iS U £ O o

O l

c *>< w O l

^ CD • ^

c o 01

3

.... u 0)

c t£

O l

sin

3

<A

0)

E o (A 3

u

c o

o « oi • ^ O l o r- 9) C ^ ^ 0)

. ?£ l —I a; <

. m ^ (A ^ 3

£ Si ' | . o ^ £ "o « 2 3 oc ^ CD •

C A ?

C VI

<A . « 3 O

X> C

o !D

c S 2 « c o <A 93 3

- 1 o .

<A <" lA

g£ § i -o o> » -M O CD

a. c C 0)

CD 3

9i U)

^ (A — o 0) V

u 5: = a> > • 0 )

CD 0) E Oj w —

S iJ ">

0> ' 7 (A OT O 3

D •- .o

0)

SJ ^ o

il > •-01 - o .c v

£ 0) U w

Is

E ^

c o 0)

a> .c (0 j 

wil

lA

^ c 0)

3

(A C

o

sti

01 O ) O l 3 (A w. 0)

u T5 C CO

r o 4-rf CD 0)

</) u

c o c gi (A 0)

• o

•o 01

ca. E o u > •

_CD

Q . = <A CO

Q 5

c

o" o Q .

U CD

w Q1 CD O tsl 0>

2 > S = 2 <A - - CA

O C3. 9)

CC

(A • CA

<A CD

t» O 0)

CO

Ol .5

01 o c CA i ; . i N

* - (A — C UI 3 V Q . . O

"Si" t/) oi.E

w (A 0 O l

0 1 c

CO C

(A Q .

-) O

jec

o h .

Q .

O

C

9) •a

>

n V

a. > * r f

<A (A 3 Cj (A

•o <A

dent

3

^

tar

(A

^

wil

> 0)

.c (A

ines

CA 3

Q Ui Si

(A 01 a > c 9)

9)

^ ro t N

arch

9i (A 01

CC

C T3

c OJ » -

^9 >-o o

OJ 3

"O <A -53

ines

se

sen

fi

CA O

3 x: .a u

(A

o O

n •o

Han

(A A O

-)

c CD

O l

c JA f "5 -o o

c CD a ,

P OJ « 01

^ ? c c o » « « to c > a C CD ^ OJ

.— > CD w

ffi _CA

01 OJ 2 «< OJ o 03 £ OJ

§ O «A

U S ID

0) •q > c

X o s:

c I/) c o

O) 3 o -a> o CD

U Q. O

> CD

T3

01 . C

3 O

. 0 (O

^ 3 01

c 01 Q. 01

c 01

c CO

J£ (A CO

o 03

O CA * - O

. 0 Q -

2. ^ 5 | |

<A CO ™

CA CO * -

-1 = S 3 -C «A 03 O 03 ^ CD . C O 03 "

iJ .tS O

03

3

0 j <

u 03

.c 0 s

• 0 03

• ^

• C OJ

• 0

^ CD 0

ffi

. CA 03

ilit

Si

(A CA 0 a.

c •5

u 03

.c u 0 )

* 01

c

CA 03

c <A 3

c m E 0

u 0

*.* _C

X a . u j c c — u i Z U u j c / )

< oc z _ z a

3 o S > • OJ £ CA C o <A 01 3

* OJ CA «J

c 5 01 M

• o X 3 OJ

k^ = CA OJ

|1 $ Q

CA CA *..

CA 0 3

> • = CD ~ •O 03

CO - C

03 a,

$ lli c QC 3J

3

03 Si

190

DAILY LESSON PLAN

Course HECE Unit Entrepreneurship

Lesson Title Entrepreneurship Class Period/Time 1st period

Essential Element(s) b(5)C. bfS'tP

OBJECTIVE(S): TLW define entrepreneurship and entrepreneur

INSTRUCTIONAL ACTIVITIES

FOCUS

EXPLANATION/ MOOaiNG

CHECK FOR UNDERSTANDING

GUIDED PRACTICE

CHECK FOR MASTERY

CLOSURE

RETEACH STRATEGIES Peer tutoring Video - Entrqjrencurship in Action

Hand out worksheet on "Home-Based Business Vennirc." This will give the students an idea of the type of business they would like to open. It is based on skills, interest, or experiences gained through school, hobbies, friends, or family. The student is suppose to circle the pictures and notions that interest them.

• Introduce a transparency with the words entrepreneurship and entrepreneur showing. Ask if anyone can give a deAnition of either. Praise if anyone knows. Uncover remainder of overhead and have students write down the defmitions of each word:

entrepreneur - a person who starts and maintains a business entrepreneurship - organizing and managing a new business

• Discuss 4 characteristics of entrepreneurs: goal-setter, risk-taker, hard worker, prepared.

• Have students get in groups of 3-4 and brainstorm advanuges and disadvantages of being an entrepreneur. Each group lists theirs on a transparency and reports to class. Students should copy 3-4 of each into their notes.

Ask questions of 1-2 students from each group (members can help): Name a characteristic of an entrepreneur. Why are entrepreneurs wiling to take the risk of going into business? Why do entrepreneurs work longer hours than employees? Name an advantage/disadvantage of being an entrepreneur.

After reviewing the advantages and disadvantages of being an entrepreneur, handout the "Checklist for Going into Business" and ask smdents if they feel they have what it takes to be an entrepreneur. Monitor as students complete.

Ask students to share results of checklist. How many would go into business for themselves? How many would select a partner? What kind of partner would you select: one just like you or one who has complemenury strengths? How many would raUier work for someone else?

Have smdents turn to back of "Home-Based Business Venmres" and write a sentence or two suting what type of business they would start if they could. If choosing a partner, what type of partner would they choose? If they don't want to start their own business, suggest they write about the kind of business where they would like to woric.

ASSIGN: interview a person you know who has started his/her own business. Write a summary of the interview and be prepared to report on Friday Ask smdents to suggest the types of questions they might ask this person. Hand out a suggested list of questions they may use.

56

INDEPENDENT PRACTICE Interview an entrepreneur.

RESOURCES Text Handouts - 2 Transparencies - 4

191

4 M A T LESSON PLAN

Course PCD

Lesson Title Turn Onto Books

Unit Rnnlcs, Tovs. Games

Class Period/Time

Essential Elementls) r(7)a. r(8) f. s(3)d

OBJECTIVE(S): TLW pracdce rules of reading to children: evaluate reading skills: create an altemarive method of storytelling

INSTRUCTIONAL ACTIVITIES

Develop you own way of relating a story -reading, telling, acting, puppets, involving child, etc. OR Write and illustrate your own children's story.

Name methods for sharing books/stories with children. View a video of a children's story or of using puppets.

Select a children's book and practice reading to someone observing the rules of reading to children.

Use an evaluation sheet to evaluate someone else's reading. Have (liem evaluate you.

Read case studies and identify what the reader did wrong when reading to a child.

RESOURCES Videos, case studies, books, art supplies,

puppets, evaluation sheets

4R

4L

3R

3L

IR

tL

2R

2L

Have students sit on floor in circle around the desk or a high stool. 1 sit on stool and read to them about choosing books for children from a college text book.

Describe and analyze how they felt. Elicit responses: too many large words. I didn't understand, boring, no pictures. I couldn't see, uncomfortable, etc.

Show video on reading to children: Winning Waords with Children

Following video, smdents hst rules for reading to children and put them in their notes. Have students demonstrate currect way to hold book. etc.

192

SHOW ME WHAT YOU LEARNED! ADD the following to the figure at right:

1.

2.

3.

4.

5.

6.

7.

8.

EARS if you listened well to the the presentation/class.

EYES if you got new "eye-deas" (ideas).

SMILE if you liked what you experienced today.

HEART if you think the speaker/I got to the "heart" of things.

NOSE if you know all the information already.

FEET if you feel motivated.

HAIR if you thought things were really "hairy" (a mess).

CLOTHES if you thought everything was covered.

JUST FOR FUN, decorate the stick figure below!

Write one or two comments (good or bad) about today's class.

Note from ttie Management: THANKS!

72

193

APPENDIX E

PORTFOLIO PROTOTYPE QUESTIONNAIRE

194

IM

Portfolio Prototype Evaluation

Thank you for agreeing to review the portfolio prototype based on the five learner-centered proficiencies adopted for Texas educators (Texas Education Agency, Learner-Centered Schools for Texas, 1994). The materials in the portfolio prototype represent the evidences that 65% of the respondents to a survey (two-round Delphi) agreed would best assess the five teaching proficiencies. The study, Developing a Portfolio Prototype for Assessment of Home Economics Preservice Teachers in Texas, surveyed home economics teacher educators, home economics student teachers, and education personnel administrators in Texas.

The format of the portfolio prototype was developed using current research on teaching portfolios. The portfolio model used (Seldin, 1991) has three major divisions: Background Information, Information from Others, and Products of Teaching.

One major theme in portfolio research/use is the reflective statement. Reflective statements in teaching portfolios allow the teacher to think back on her/his particular teaching experiences and gain insight and meaning from those experiences. Reflective statements are considered essential for understanding the portfolio's purpose and contents and are viewed as important as the item of evidence itself.

Individual items of evidence in the portfolio prototype are not intended to be the best examples of their kind but are intended to represent materials typical of teaching. Portfolio evidences are to be evaluated only on the degree to which the types of materials in each subdivision would be useful for providing evidence of one or more teaching proficiencies. Review the portfolio in your role as personnel administrator, teacher educator, or recent home economics student teacher.

After you become familiar with the contents of the portfolio, please answer the attached evaluation sheet. The evaluation sheet is ordered in the same sequence as the portfolio. Any subdivision of evidence may be valuable for assessing more than one teaching proficiency. In that case, you could have two (or more) of the same rating scale number beside a single portfolio evidence.

Thank you for participating in this study.

Retum portfolio in enclosed mailer by

Study conducted by Lora Ann Neill. 204 E. 91st Street, Odessa, Texas 79765, (915) 363-8648. ®1996 by Lora Ann Neill. All rights reserved.

195

alua

tion

>

otyp

ro

t

Z4

Port

folio

60 c

ente

red

teac

h!

V «

re le

amer

an

k sp

ace

I C C

•5 M

of

opri

ea

ch

appr

eou

;ful

ness

in

mea

suri

n 4,

3.

2, o

r 1)

in

di

ft as

to

its u

s(

resp

onse

s (5

, fo

lio e

vide

nces

o

art.

Rec

ord

you

port

ec

h Io

n of

p

of d

i

.ss o

ite e

ach

subd

ii tll

y ac

ross

the

s i (« e

Usi

ng t

he s

cale

bel

ow, e

v pr

ofic

ienc

ies

show

n ho

riz

r

no u

se

O

II « M

2 =

of

little

use

ci

ded

ind

e

II m

ful

= u

se

^

sefii

l

9 >

5 =

ver

sai3

z bd ^ U

PR

O

ED

TE

AC

HIN

G

R-C

EN

TE

R

z Oi < Ed

ra O

2 | . iS

The

oall

on

stra

0

imp

mai

nla

pers

o

< z « 5 ' S-o " ztrii.8HJg.si

SS

IO

OP

M

IS a

re

erde

d su

cce;

en

t to

ssio

n,

nal

eth

PR

OF

E

DE

VE

L

teac

her,

i pr

aditi

on

stud

ents

' co

mm

itm

the

prof

e pr

ofes

sio

inte

grity

.

hile

A

TIO

N:

W

ocat

e fo

r al

l sc

hool

, th

e ra

tes

effe

ctiv

e in

terp

erso

nal

ikill

s.

NIC

la

dv

dthe

no

nst

land

tio

n

=»:; s s s s

MM

Ig

as

ents

he

rd

essi

o m

uni

CO

ac

tii

Slud

te

ac

prof

co

m

FY

IN

L

LE

NC

E:

The

re

spon

ds a

ppro

pria

tely

-se

gro

ups

of le

arne

rs.

EQ

UI

EX

CE

te

ache

r to

div

ei

2-s . fc

Toe

[n

mu

ivel

y pl

ans

sses

no

lof

INS

TR

UC

TIO

N:

a le

amer

-cen

tere

d co

th

e te

ache

r co

llabo

rat

iden

tifie

s ne

eds;

and

im

plem

ents

, an

d as

se

inst

mct

ion

usin

g te

ch

and

othe

r re

sour

ces.

E:

The

teac

her

-aw

s on

a r

ich

of c

onte

nt,

echn

olog

y lo

an

d m

eani

ngfti

l nc

es f

or a

ll

KN

OW

LE

DG

po

sses

ses

and

di

know

ledg

e ba

se

peda

gogy

, an

d 1

prov

ide

rele

vant

te

amin

g ex

peri

e st

uden

ts.

POR

TFO

LIO

E

VID

EN

CE

S (S

ubdi

visi

ons)

saoisiAKI

Intr

oduc

tion

Philo

soph

y of

Tea

chin

g

Prof

essio

nal

Act

iviti

es

Hon

ors

and

Aw

ards

Cer

tific

atio

ns

a

Uni

vers

ity C

lass

Pro

jec

Rel

ated

to

Tea

chin

g F

ield

aopsauojni pnnojS^vg

Eva

luat

ions

fro

m H

igh

Sch

ool

Stud

ents

Let

ters

fro

m H

igh

Sch

oo

Stud

ents

sjsqio mojj nofimuojai

Un

il/L

esso

n P

lans

Tea

cher

-dev

elop

ed

Inst

ruct

iona

l M

ater

ials

Tea

cher

-dev

elop

ed

Eva

luat

ion

Mat

eria

ls

1

Cre

ativ

e A

ctiv

itie

s

Tec

hn

olog

ical

Act

ivit

ie

Sam

ples

of

Stud

ents

' W

ork

Par

ent

Con

tact

s

Wor

king

wit

h Sp

ecia

l N

eed

s St

uden

ts

8a}q3B3x JO spnpojj

196

7LEASE ANSWER THE FOLLOWING QUESTIONS ABOUT THE PORTFOLIO PROTOTYPE.

1. How would you use a portfolio similar to the prototype? (Please check all that apply to your use.) for assessing teacher proficiencies in my school/school district to provide evidences of my own teaching proficiencies in an interview situation as a final grade for a course with high school students (as a graded project) with high school smdents (personal development only) for hiring teachers (provides enough information to hire without an interview) for hiring teachers (review information before or during an interview) for my own personal development for program development for program evaluation as a formative evaluation of student teaching as a summative evaluation of smdent teaching for appraisal of my teaching skills as a basis for writing recommendations for teachers/smdent teachers

Other

2. Based on the use(s) you checked in question 1, how would you rate the overall usefulness of the portfolio prototype? very useful useful undecided

of little usefulness of no use

3. Please assess the physical/fimctional characteristics of the portfolio prototype. (Check all that apply.) The portfolio prototype

has adequate description of portfolio items. has dividers clearly marked. is easy to review. is logically organized. has a professional appearance. won't easily come apart.

COMMENTS:

4. List any items you feel should have been included in the portfolio prototype. Please explain which proficiency each item would evideiKC.

5. What items could have been omitted firom the portfolio prototype?

6. What suggestions do you have to improve die portfolio prototype?

A. Appearance

B. Organization

197

APPENDIX F

USEFULNESS OF EXAMPLES OF EVIDENCES IN

ASSESSING MASTERY OF EACH PROHCIENCY

198

Table F.l Usefulness of Portfolio Prototype Evidences in Assessing Mastery of Leamer-centered Knowledge. (A/ = 15)

The teacher possesses and draws on a rich knowledge base of content, pedagogy, and technology to provide relevant and meaningful leaming experiences for all students.

scale: 1 {of no use) to 5 {very useful)

Div

isio

ns

Bac

kgro

und

Info

rmat

ion

Info

rmta

ion

Prod

ucts

of

Tea

chin

g

Portfolio Evidences

Introduction



Honors and Awards

Certifications

University Class Projects Related to Teaching Field


Letters from High School Students

Unit/Lesson Plans

Teacher-developed Instructional Materials

Teacher-developed Evaluation Materials

Creative Activities


Samples of Students' Work

Parent Contacts

Working with Special Needs Students

Mean for All

Respondents

3.55

3.9

3.55

4.1

4.5

4.23

4.36

3.55

4.9

4.82

4.6

4.7

4.67

4.44

3.0

4.3


in =4) Mean

4.0

3.0

4.0

4.0

4.67

4.67

4.0

4.0

5.0

5.0

4.67

5.0

5.0

4.67

2.5

4.5

Teacher Educators

(n = %) Mean

2.6

3.8

2.8

3.75

4.0

3.86

4.2

2.6

4.75

4.8

4.5

4.25

4.5

3.75

2.0

4.0

Personnel Admiaistrators

(n = 3) Mean

4.67

4.67

4.33

4.67

5.0

4.67

5.0

4.67

5.0

4.67

4.67

5.0

4.67

5.0

4.67

4.67

199

Table F.2 Usefulness of Portfolio Prototype Evidences in Assessing Mastery of Leamer-centered Instruction. (A = 15)

To create a leamer-centered community, the teacher collaboratively identifies needs; and plans, implements, and assesses instmction using technology and other resources.


Div

isio

ns

Bac

kgro

und

Info

rmat

ion

Info

rmat

ion

Prod

ucts

of T

each

ing

PortfoUo Evidences

Introduction



Honors and Awards

Certifications




Unit/Lesson Plaiis



Creative Activities



Parent Contacts


Mean for All

Respondents

2.89

3.67

3.11

3.0

3.8

4.5

4.36

3.45

4.83

4.62

4.75

4.77

4.83

4.69

3.0

4.6


(n = 4) Mean

3.5

4.5

3.5

3.0

4.5

4.5

3.5

4.5

5.0

4.67

4.67

5.0

5.0

4.67

1.5

5.0

Teacher Educators

(n = 8) Mean

1.5

2.75

2.0

2.0

3.2

4.2

4.33

2.5

4.67

4.43

4.67

4.71

4.83

4.0

2.5

4.2


(n = 3) Mean

4.33

4.33

4.33

4.33

4.33

5.0

5.0

4.67

5.0

5.0

5.0

4.67

4.67

4.67

4.67

5.0

200

Table F.3 Usefulness of Portfolio Prototype Evidences in Assessing Mastery of Equity in Excellence for All Leamers. (A = 15)

The teacher responds appropriately to diverse groups of leamers.


Div

isio

ns

Bac

kgro

und

Info

rmat

ion

Info

rmat

ion

Prod

ucts

of

Tea

chin

g

PortfoUo Evidences

Introduction

Philosophy of Teachiiig


Honors and Awards

Certifications


Evaltiations from High School Students


Unit/Lesson Plans



Creative Activities



Parent Contacts


Mean for AU

Respondents

2.56

3.22

2.44

2.33

2.78

3.33

3.8

3.67

4.44

4.78

4.75

4.27

3.6

3.9

3.9

4.67


(n = 4) Mean

2.0

3.0

3.5

3.5

3.0

3.5

4.0

3.5

4.67

5.0

5.0

5.0

3.67

5.0

4.0

5.0

Teacher Educators

(n = 8) Mean

2.0

2.25

1.5

1.25

2.0

2.25

3.5

2.75

3.67

4.5

4.33

3.6

2.75

2.5

3.0

4.5


(n = 3) Mean

3.67

4.67

3.0

3.0

3.67

4.67

4.0

4.0

5.0

5.0

5.0

4.67

4.67

4.67

5.0

4.67

201

Table F.4 Usefulness of Portfolio Prototype Evidences in Assessing Mastery of Leamer-centered Communication. (A = 15)



Div

isio

ns

Bac

kgro

und

Info

rmat

ion

Info

rmat

ion

Prod

ucts

of

Tea

chin

g

Portfolio Evidences

Introduction



Honors and Awards

Cotifications




Unit/Lesson Plans


Teadier-developed Evaltiation Materials

Creative Activities



Parent Contacts


Mean for AU

Respondents

3.78

3.89

3.7

3.33

2.89

3.6

3.36

3.27

3.75

3.89

4.13

3.56

3.73

2.78

4.25

3.55


(n = 4) Mean

5.0

5.0

4.33

2.5

2.5

3.5

4.0

4.0

2.5

2.5

2.5

3.5

1.5

2.0

3.5

3.0

Teacher Educators

(n = 8) Mean

2.5

2.75

2.75

2.5

2.5

4.0

3.2

3.0

3.33

3.75

4.33

3.75

4.0

2.25

4.29

3.0


(« = 3) Mean

4.67

4.67

4.33

4.0

3.67

4.33

3.0

4.0

5.0

5.0

5.0

4.67

4.67

4.0

4.67

5.0

202

Table F.5 Usefulness of Portfolio Prototype Evidences in Assessing Mastery of Leamer-centered Professional Development. (A = 15)

The teacher, as a reflective practitioner dedicated to all students' success, demonstrates a commitment to leam, to improve the profession, and to maintain professional ethics and personal integrity.


Div

isio

ns

Bac

kgro

und

Info

rmat

ion

Info

rmat

ion

Prod

ucts

of

Tea

chin

g

Portfolio Evidences

Introduction



Honors and Awards

Certifications



Letto^ from High School Students

Unit/Lesson Plans



Creative Activities



Parent Contacts


Mean for AU

Respondents

4.17

4.27

4.82

4.5

4.31

3.7

4.0

3.56

3.88

3.67

3.75

3.22

4.1

3.11

3.5

3.7


(n = 4) Mean

4.67

4.0

5.0

4.0

4.67

4.0

5.0

5.0

4.5

4.5

3.5

4.5

4.5

4.5

4.0

4.67

Teacher Educators

(n = 8) Mean

3.5

4.17

3.83

4.57

4.0

2.75

3.0

2.25

3.0

2.75

3.33

2.0

3.8

2.0

2.75

2.5


(/' = 3) Mean

5.0

4.67

4.67

4.67

4.67

4.67

4.67

4.33

4.33

4.33

4.33

4.0

4.33

3.67

4.0

4.33

203

of home economics preservice teachers in texas

Documents

Transcript of of home economics preservice teachers in texas