Using Embedded Assessments to Measure Student Learning

OutcomesHarper College

Outcomes Assessment

IOAC Assessment Conference and Share Fair

March 8, 2013

Previous Workshops Why do we assess, and how do student learning outcomes help us assess?

How do we begin to develop student learning outcomes?

What are the characteristics of well-written outcomes?

Don’t worry if you don’t know the answers to these questions, just contact me after this session for more information.

Learning OutcomesUpon completion of this workshop, participants will be able to: Explain how embedded assessment can be beneficial when measuring student learning outcomes.

Identify what course activities might be used to measure specific student learning outcomes.

(Be able to) create an assessment plan using embedded assessments.

Today’s Agenda Background: Where are you on your assessment journey?

What are embedded assessments, and what are the advantages of using them?

Which course activities should be used for embedded assessment?

How are course activities linked to student learning outcomes?

(Create an assessment plan using embedded assessment techniques.)

Where are you on your assessment journey?

What are embedded assessments?Embedded assessments… Measure outcomes through work your students already do in your course

May be a project, a set of questions included in a final exam, a final paper, or anything that is regularly used to evaluate the students in the class

What are the advantages of using embedded assessments? Uses items you are already asking your students to complete

Lessens faculty resistance; promotes positive dialogue and shared responsibility

Can be graded separately from the assessment evaluation itself, if faculty desire

Students generally understand the value (even if only to their own grades) of embedded assignments and take them seriously

What are the advantages of using embedded assessments? Impact on work loads minimized

Saves time because assignments will be (or are already) required for the course

Usually easier to coordinate and score than are new/outside assessments

What are the disadvantages of using embedded assessments? Coordination may be difficult, especially for larger departments, departments with large numbers of adjuncts, or departments that do not usually work together on individual courses.

May be seen as limiting faculty autonomy within the course.

Note: May still be easier and less intrusive than other forms of assessment.

Which of my course activities can be used for embedded assessment? Course-level

Questions embedded on the final exam Final essays/papers scored using a common rubric Projects scored using a common rubric Existing assignments or projects that all students in

the course complete Program-level

Capstone experience Internship evaluations Licensure exams Surveys or interviews (graduate, employer) Portfolio analysis Existing assignments or projects that all students in

the program complete …others you are thinking about?

Which of my course activities can be used for embedded assessment?

Nearly anything your students do in your course can be used for assessment For course-level outcomes, use an item near the end of the course

For program-level outcomes, use an item from a course that is near the end of the program (or after the program, i.e. externship evaluations)

Which of my course activities can be used for embedded assessment? If you are coordinating assessment across several sections of a course: Start by asking if there is an assignment or exam that all (or nearly all) instructors use

Even if the items are graded differently or have different questions, the department can find or create common pieces to use in assessment• Embed common exam questions• Use a rubric for assessment but allow instructors to continue grading using their own system, if desired

Where do I start? How do I link these assignments to the course outcomes?Please call the Assessment Office at any time you would like additional help, advice, etc.1. Get as many faculty involved in this

process as possible. 2. Review the course (or program) learning

outcomes (and ensure all faculty teaching the course have the same version of those outcomes)

3. Discuss what common items the faculty teaching the course/program already have students complete (final exam, paper, project). They do not have to be completely common. You

may simply find that all instructors do a paper at the end of the course, but each paper is different.

Where do I start? How do I link these assignments to the course outcomes?4. Determine which outcomes might be measured

using those common items. Do you have an outcome related to

communication? Perhaps you also have an assignment, presentation, or test question that can measure written or oral communication through a common rubric.

Note: if a common rubric is used and questions are similar, the prompts would not necessarily have to be the same for this item.

Do you have an outcome related to content knowledge? Embedding a set of questions on the final exam may work well, even if individual instructors maintain the majority of their own exam questions.

Where do I start? How do I link these assignments to the course outcomes?4. Determine which outcomes might be measured

using those common items. Do you have an outcome related to critical

thinking? Again, a rubric used to measure a common assignment or essay question may work well here.

In this case, the prompts may have to be the same, or at least share similar qualities, to elicit similar responses from the students.

Wouldn’t it be wonderful to measure more than one outcome through one item? You could use a single rubric to measure communication, critical thinking, and content knowledge, if you had an appropriate assignment or test question embedded in the course.

Once you have aligned assignments/test questions/performances with course outcomes

5. Revise prompts and questions as needed. Depending on the outcome and assessment

method, some items may need to be adjusted in order to ensure students in each section are receiving similar instructions.

Item should also count toward a similar percentage of the students’ grades (from section to section).

6. Develop a rubric or other common scoring mechanism. It will be helpful to train faculty members

on the use of the rubric. Remember, faculty members do not have to use the assessment rubric for their own grading.

Discuss the possibility of sharing the rubric with students. This improves their buy-in as well as their understanding of how the assignment is being assessed.

Once you have aligned assignments/test questions/performances with course outcomes

7. Schedule the assessment, with plenty of time to allow faculty to adjust their syllabi, assignments, exams, etc. where needed.

8. Assess, collect and record results, and discuss use of results.

9. Start the assessment cycle again, incorporating what you learned.

Please call the Assessment Office at any time you would like additional help, advice, etc.

Examples of Embedded Assessments at Harper Speech (Course-Level)

Did you see Margaret Bilos’s poster? Assessing persuasive speeches (already done in every section of Speech 101)

Take video recordings of students Faculty members randomly assess the speeches using a common rubric (with at least three evaluators per speech)

Examples of Embedded Assessments at Harper Art (Course-Level)

Faculty members are developing an assessment of multiple learning outcomes in a single course, based on an assignment they were already using

The assignment and rubric are being adjusted, but it is an joint process among faculty members who teach that class

The instructor for the course will assess using the rubric, then the assignment will go to a second reader (and a third, if necessary)

Examples of Embedded Assessments at Harper Math (Course-Level)

Embedding common questions on Math 103 final exam

Questions will be written to target learning outcomes, but the exam itself is an embedded part of every Math 103 course

Examples of Embedded Assessments at Harper Certified Nurse Assistant (Program-Level) Use results of competency exams that are already required by the state

Many other career programs use similar structures, when accreditors, licensing bodies, etc. require exams

Can delineate different areas that need additional support or attention, based on the results of the exams

Examples of Embedded Assessments at Harper Law Enforcement and Justice Administration (Program-Level) Use internship evaluations completed by local agencies

Good for increasing understanding of students’ capabilities outside the classroom

External evaluators may be more objective in their appraisal of student abilities and achievements

Examples of Embedded Assessments at Harper Fashion (Program-Level)

Evaluate capstone projects for both Design and Merchandising

Design students are evaluated through the Jury Show, which involves external evaluation

Can assess multiple outcomes through a single capstone project

Want to know more?

Full workshop will be offered through Professional Development (will include hands-on work with your course work and outcomes)

One-on-one, group, department, and division meetings are always available

Please direct all questions & comments toFaon GrandinettiManager, Outcomes Assessment(847) 925-6356fgrandin@harpercollege.edu