Assessment Update - Cisco

1© 2003 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID

Assessment Update

222© 2005 Cisco Systems, Inc. All rights reserved.ADI Update Summer 2005

Today’s Topics

• Introduction

• Update on developments in the past year

• What we have heard, how we are responding, innovations to come.


Who are We?

• The Academy Assessment team is the group that creates, deploys and monitors the assessment content, and sets the policies for use of the Academy exams.

• The Academy Assessment team is part of the Assessment Development and Innovation team in Cisco Systems.

FACTThe Academy program has delivered over 42 million examsinto classrooms. In 2005 average delivery is over 800,000 exams a month


Our Team Mission

• Support the learning and decision making of individuals and organizations by accurately describing learner knowledge, skills, and abilities.


Accomplishments for Ongoing Improvements

555© 2003, Cisco Systems, Inc. All rights reserved.


Accomplishments for Ongoing Improvements

• Cisco Network Simulator, CNS, is now available to students in CCNA 2• Localized the entire CCNA suite of courses (including the Bridge into eight

languages)• New processes lead to rapid correction of errors and regularly scheduled

improvements in content (Please see FAQ 2188 and errata FAQs)• Created new CCNA Practice Finals for each course and provided multiple

forms of the CCNA finals• Skills Review Exams introduced in PNIE and IT Essentials II• Added multiple Certification Practice Exams for INTRO, CCNA and one for

each CCNP course• Added Academy CCNA Certification Practice Exams to the free CCNA Prep

Center site on www.cisco.com, http://forums.cisco.com/eforum/servlet/PrepCenter?page=main

• Completed new versions for all exams in PNIE, FNS, CCNA Bug Fix, IT Essentials II, Wireless and CCNP 1 and 2

• Exam viewer


What We Have Heard andWhat are We Doing in Response



Question We Have Heard

• Are Academy exams aligned with the certification and how do you decide what is tested in the academy exams?

• What can you do to support our testing of hands-on skills?

• How can security be improved in the Academy exams?

• How the Academy assessments help the Academy program differentiate itself from other programs?

• How can instructors provide input?


Are Academy exams aligned with the certification and how do you decide what is tested in the Academy exams?



Are Academy Assessments Aligned with the Certification?

For Cisco certifications

• Key differences occur in terms of delivery, tasks types, and testing conditions.

• The same design documentation is available to both teams, certification and Academy. As a result the content is similar.


Common Task Type, Rich Media Tasks Added to Academy Assessments


Common Task Type, Rich Media Tasks Added to Academy Assessments

Here are some task types you can expect to see in Networking Academy exams over the next year.


Common Design Documentation

Final Exam claims are linked to the certification claims.Tasks designed using final exam claims and component skills provide summative feedback.


Common Focus of Content

The Domain Analysis Table and component skills help to focus the content on real world skills.


What can you do to support our testing of hands-on skills?



Cisco Network Simulator Skills Exam Pilot is Now Live in CCNA 2!

Provides more opportunities for skills exams using a real task, performance based, online assessmentAutomatically scoredCompletely integrated with exam activation and grade book


Cisco Network Simulator – How to Setup

These steps must be completed on each workstation. • Step 1 - Verify that the Java Run-Time Environment (JRE) is

installed.

• Step 2 - Download and install the JRE, if necessary. JRE can be downloaded from the Sun web site or follow the directions found at, http://sandles.unicon.net:200/installation-windows-ie.html .


Cisco Network Simulator – How to Setup

These steps must be completed on each workstation.

• Step 3 - Verify network access to CNS Server. – Ports 80 and 443 must be open to IP addresses 66.179.77.137 and

56.179.3.206.

– From each workstation, the computer, the computer must be able to communicate with the server named "skills.netacad.net" on the ports 80 and 443.

• If you do not know if ports 80 and 443 are open, proceed to the next step, and the url will test these ports.

• Step 4 - Validate installation. Click on, http://sandles.unicon.net:200/ValidationStart.action. If you are able to interact with the router simulation and click successfully on the "Submit Scenario" button, the workstation and browser areready to go!


Cisco Network Simulator Where to Get Help?

• I will add screen shots of the tools page and faqsnext week after it goes live.


How can security be improved in the Academy exams?



How Can Security be Improved in the Academy Exams?

• Flash theme improves delivery security.

• Currently on the 801 Certification practice exams in CCNA 4. Plans are to deliver other exams using this theme in the upcoming year.


How Can Security be Improved in the Academy Exams?

• Large item pools and multiple forms of the exams should help exam security.

Two forms of the finals are now available in CCNA 1 –4 English.


Field Testing Ensures Equated Forms

• Field testing will occur for CCNA content initially.• Unscored questions that look like all other

questions will be added to exams to collect data –they do not affect the student’s score –only a small number of tasks will be added to an exam

• Benefits of using field testing include: –provides data on tasks before they are used on scored exams–facilitates the development of higher quality assessments–allows for the creation of balanced, multiple forms of an exam


How the Academy assessments help the Academy program differentiate itself from other programs?


Differentiation from Other Online Programs


• Tight connection of curriculum exams with certification exam

• Extensive design process• Constant performance monitoring through

statistical analysis • Refreshes based on performance monitoring and

instructor input• Different types of exam to fulfill different

instructional needs


Different Exams for Different Instructional Needs Differentiate the Academy Program

Category QuizSkills Review

Exam Module Exam Practice Final Final ExamPractice

Certification Voucher

Organizational Purpose:

Provide practice and feedback to student

Feedback to student and instructor

Provide practice and feedback to student, instructor and administrator


Provide feedback to student and teacher


Provide minimum level for voucher

Instructional Purpose: Formative Summative

Formative or Summative Formative Summative Student feedback None

Grain Size of Feedback:

Task and option level

Task and subscore level

Task level and overall proficiency



Task level and overall proficiency Overall proficiency

Grain Size of Claims and Tasks: Small to medium Small to medium Small to medium Medium to high Medium to high Medium to high Medium to high

Level of Task Complexity: Varies Varies Varies High High High High

Level of Content Security: Low Low to medium Low to medium Low to medium High Low Very High

• Assessments provided for 35 courses in English

• Typically 20 – 30 assessments per course

• Courses in nine languages

• Constant maintenance and refresh must occur to ensure exams stay relevant and usable


How can instructors provide input?



How Can Instructors Provide Input?Method 1

• Through the Help Tool, select Assessment Quality.– Expect a response within 5 days of receiving issue.

– Exams are typically refreshed every three months or less.

– An Errata Sheet is created and uploaded to the FAQs.

– Refresh schedule for exams is posted to FAQ 2188.



Provide feedback before assessments go live. Here is how:

– Instructor are asked to volunteer as External Reviewers. If you are interested, please let us know!

– External Reviewers provide feedback on tasks before they go live.

– The Assessment Team reviews each comment and edits the item accordingly.



Participate in the Online Assessment Forums and Chat area in Academy Connection Community area.


How does this play out in the Networking Academies for the upcoming year?



Important Points

• You should see improved exam design and the accompanying design documentation.

• Multiple forms of the final and module exams will start being available for some courses.

• We need your help Piloting the On-line skills based exams via CNS

• We depend on your input. Please keep it coming!


Q and A


Assessment Update - Cisco

Documents

Transcript of Assessment Update - Cisco