GameAbove College of Engineering & Technology Annual ...
-
Upload
khangminh22 -
Category
Documents
-
view
3 -
download
0
Transcript of GameAbove College of Engineering & Technology Annual ...
1
GameAboveCollegeofEngineering&TechnologyAnnualAssessmentReport
2020-2021
1. AssessmentProcessIntroduction
TheGameAboveCollegeofEngineering&Technologycurrentlyhas29academicprogramsthatarelocatedinfourdifferentschools:SchoolofEngineering,SchoolofInformationAssurance&AppliedComputing,SchoolofTechnology&ProfessionalServicesManagement,andSchoolofVisual&BuiltEnvironments.Thecollegehasseenagrowthinmanyofitsprogramsasaresultofchangesmadetotheprogramsinresponsetonewandemergingtechnologies,industrydemands,andstudentexpectations.MostprogramhaveAdvisoryBoardswhosemembersconsistofcurrentstudents,faculty,alumni,andindustryexperts.TheAdvisoryBoardsarekeytohelpingprogramsstayabreastoftheneedsofstudentsandindustrythroughtheirfeedbackandrecommendations.Twoprimarygoalsofthecollege,whichtiecloselywiththeassessmentprocess,aretoadvancetheoverallqualityofacademicprogramsandtoincreaserecognitionforthecollegebypursuingaccreditationfromexternalaccreditationbodies.
AssessmentProcess
TheGameAboveCollegeofEngineering&Technology’sAssessmentCommitteeismadeupoftheAssociateDeanandanelectedfacultymemberfromeachofthefourschools.TheAssessmentCommitteeinitiallymeetsatthebeginningofeachfallsemestertodiscussthepreviousyear’sassessmentandtoidentifychangesthatmaybeneededduringtheupcomingyearregardingthecollegeassessmentprocessand/ortemplate.TheCommitteemembersserveasfacilitatorsbetweentheAssessmentCommitteeandthefacultyineachoftheirrespectiveschoolsregardinganychangesorrecommendations.Allcommitteeandfacultymeetingsfor2020-2021wereheldvirtuallyviaZoomduetorestrictionsasaresultofCOVID-19.TheDirectorsforeachschooldeterminewhowillconducttheassessmentforeachoftheirprograms.Theassessmentistraditionallyconductedbytheprogramcoordinator;however,theDirectormaychoosetoselectanotherseniorfacultymemberduetoextenuatingfactors.TheDirectorsofeachschoolareresponsibleforinitiallyreviewingandapprovingtheassessmentreportssubmittedbytheprogramcoordinatorsorfacultyfromtheirschoolbeforetheyaresubmittedtotheAssociateDean.
2
AccreditationProcess
Assessmentsareakeyfactorforallaccreditationbodiesandmeetingtheirstrictmandatesandcriteriaarenecessarybeforeanyprogramisaccredited.Becauseofthediversityofthecollegeprograms,thisbecomescomplexbecausetherearecurrently9differentexternalaccreditationbodiesthatreviewandmonitor16programsinthecollege.Figure1identifiesthecurrentaccreditationbodiesandprogramsthatareeithercurrentlyaccreditedorintheapprovalprocess.Theaccreditationsrequirethatprogramsandcertaincourseswithinthoseprogramsbeassessedorreviewedyearlyusingmetricsidentifiedbytheaccreditationbodyorbytheprogram(ex:enrollment,sustainabilityrate,graduationrate,resultsfromsurveysdistributedtocurrentstudents,graduatingstudents,andalumni).
Figure1showstheaccreditationbodiesandprogramsthatareeithercurrentlyaccreditedorintheapprovalprocess.
• ABET-EAC:AccreditationBoardforEngineering&Technology–EngineeringAccreditationCommission:
¨ MechanicalEngineering,B.S.¨ Electrical&ComputerEngineering,B.S.
• ABET-ETAC:AccreditationBoardforEngineering&Technology–Engineering&TechnologyAccreditationCommission:
¨ MechanicalEngineeringTechnology,B.S.¨ ProductDesign&Development,B.S.¨ ElectronicsEngineeringTechnology,B.S.¨ ComputerEngineeringTechnology,B.S.
• ABET-CAC:AccreditationBoardforEngineering&Technology–ComputerAccreditationCommission
¨ InformationAssurance,B.S.¨ InformationTechnology,B.S.
• ACCE:AmericanCouncilforConstructionEducationAccreditation:¨ ConstructionManagement,B.S.
• CIDA:CouncilforInteriorDesignAccreditation¨ InteriorDesign,B.S.
• NASAD:NationalAssociationofSchoolsofArt&Design¨ Simulation,Animation,andGaming,B.S.¨ InteriorDesign,B.S.
• AABI:AviationAccreditationBoardInternational¨ AviationFlightTechnology,B.S.¨ AviationManagementTechnology,B.S.
• ACPHA:AccreditationCommissionforProgramsinHospitalityAdministration¨ Hotel&RestaurantManagement,B.S.
• ABA:AmericanBarAssociation¨ Paralegal,B.S.
Figure1:AccreditationbodiesforGACETprograms(currentlyaccreditedorintheapprovalprocess)
3
GACETAssessmentTemplate
Figure2showsthetemplateusedbyGACETprogramsforthecollegeassessmentreport.
Figure2:GACETAssessmentTemplate
4
2. Specificexamplesofimprovementsmadetocourses,programs,instructionalapproach,etc,:
A. ElectricalandComputerEngineering–EECE342TheEECEprogramisanABETaccreditedprogramintheSchoolofEngineering.Lastyear,severalrecommendationsweremadetotheprogramandcourses.Basedontheassessmentandrecommendations,thefollowingactionsweretakento‘closetheloop’:
• PrerequisiteschangesweremadetoEECE341andEECE371.Thechangeswereupdatedtobetterfitthecourselevel.
• EECE400wasmovedfrombeingofferedfallsemestertowintersemester.Basedonstudentfeedbackandtherolloutoftheclasses,itwasfoundinpreviousassessmentsthatEECE400wasofferedinvarioussemesterswithoutthoughttowhenitwasneededfortheirseniorfinalexam.Thedecisionwasmadetomakethisa10-weekcourseandofferitwintersemesterjustpriortothefinalexam.
• Lastyear’sassessmentfoundthatstudentswhotookarequiredclass(ME100orET100)weremissingkeyelementsneededfortheEECEprogram.Therefore,anewcourse(EECE100)wascreatedtoreplacetheothercourses.Thus,moreEECErelatedcoursecontentcanbeintroducedintothecourseforstudents.
Thesechangesweremadebasedonpreviousassessments.However,theywerechangedduringAY21andtheprogramhasnothadanopportunitytoassesstheoutcomesofthesechanges.Theprogramusesembeddedmaterials,suchasdesignmaterialsandsurveyresultstoassistwiththeirassessment.Theassessmentmaterialshavecorrespondingperformancecriteriathatmustbemetforeachcourse.Notmeetingtheperformancecriteriameansthatappropriatechangesneedtobemadetoimprovethestudents’achievementofthatSLO,suchastheexamplesgivenabovefor‘closingtheloop’.TheembeddedassessmentisusedtoevaluateselectSLOsinthecoursecontent.Eachperformanceindicatorisratedasexemplary,satisfactory,developing,andunsatisfactory.Theperformancecriteriaforacourseindicatesthattheperformanceindicatorshouldscorenolessthan75%ofstudentstotheSatisfactorycategoryorabove.Thisyear,theprogramassessedEECE342.AllsevenSLOsforthecoursewereassessed.TheresultsshowedthatthestudentsmetorexceededtheperformancetargetforeachareaotherthanSLO#4:Anabilitytorecognizeethicalandprofessionalresponsibilitiesinengineeringsituationsandmakeinformedjudgements,whichmustconsidertheimpactofengineeringsolutionsinglobal,economic,environmental,andsocietalcontexts.Thestudentsscored60%andtheperformancetargetwas75%.Thefacultyindicatedthatthiscontentmaynothavebeenadequatelycoveredtotheextentnecessaryintheclass,assignmentsmaynothavebeenrelevanttothatSLO,orstudentsmaynothaveunderstoodhowthecoursematerialtiedtotheSLO.Theprogramfacultywillneedtoaddressthiswhentheyreturninthefalltoseehowchangesshouldbemadetoaddressthis.Basedonthescoresandfeedbackfromthestudents/faculty,thefollowingactionswererecommended:
• Changetheseniordesigncoursefromonesemestertotwosemesterstoallowstudentsmoretimetocompletetheprojectandincorporatetheelementsfromtheseclassesintothefinalproject.
5
• Intheseniordesigncourse,moreemphasisneedstobeplacedonengineeringstandardsanddesignconstraintsandembedthecontentfromthiscourseintothefinalproject.Thus,thefinalprojectwouldhaveascaffoldingeffectfrompreviouscourses.
• TurnEECE480intoawritingintensivecourse.Currently,studentsarerequiredtotakeageneralSET350WcourseanditisfeltthattheEECEstudentswouldbenefitmorefromawritingintensivecoursethatfocusesonthecontent,material,andassignmentsrelevanttotheEECEprogram.
B. InformationAssurance&CyberDefense(IA/CD)–IA385IA385isaDatabaseandWebApplicationSecurityclassofferedtojuniorlevelstudents.Thecourseemphasizesdatabaseadministrationandprotectionandincludesinstructiononequipmentandprocedures.Thisyear,theyassessedthefollowingSLOs:
• Abilitytoanalyzeaproblem,andidentifyanddefinethecomputingrequirementsappropriatetoitssolution.
• Abilitytofunctioneffectivelyonteamstoaccomplishacommongoal.• Abilitytousecurrenttechniques,skills,andtoolsnecessaryforcomputingpractice.
Theassessmentmethodusedwasbasedondifferentdirectassessmentmethodssuchasindividualhands-onlabprojects,exams,andquizzeswithdifferentweightofvaluewereusedtoassessstudentslearningoutcomeinthisclass.Eachoftheseassessmentmethodsaimedtoevaluatestudents’learningoutcomesthatinclude:Students’abilityto1.Defineandidentify:thenatureofdatabaseandinformationsystemssecurity;thecharacteristicsofvirusesandhowtheyinfiltratesystems;securitythreats;thecharacteristicsofvirusesandhowtheyinfiltratesystems;specifictypesofoperationalsecurity;theinformationsecuritylifecycle;andthemultilayerednatureofsecurityarchitecture.2.Defineadatabaseandidentifythebasiccomponentsofadatabasemanagementsystem.3.IdentifyanddescribethearchitectureofOracle,MySQL,andMicrosoftSQLServer.4.SecuretheinstallationandconfigurationofMySQLserver,MicrosoftSQLServer,andOracle.5.Implementauthentication,authorization,usermanagement,password,rolesandprivilegeswithSQLServer,MySQL,andOracle6.UnderstandaSQLinjectionandidentifyhowinjectionsareexecuted,aswellasdefensestrategiesagainstSQLinjectionexploits.7.Understandthesecurityanddatabaseauditingprocesses.8.Understandthemethodologyusedtoperformasecuritytest.
Exam I 20% Exam II 20% Quizzes 10% Labs 30% Presentation 20% Total 100%
Thecoursematerialswereprioritizedandcondensedinthislab-intensiveclass.Whileexamsandquizzesassessedstudents’theoreticalcomprehensioninthesubjectmatter,thelabassignmentsassessedtheirpracticalhands-onlearning.Thefinalprojectwassuitablemeasurementtoassessstudents’technicalwritingaswellasteamworkachievementoflearningobjectiveinthisclass.
6
Therubricandresultsoftheassessmentareshowninthefollowingtable:
TheassessmentfindingsindicatedthattargetforInformationAssuranceprogramwasthat80%ofthestudentsshouldbeabletodemonstratecompetencywith80%accuracy.TheoveralloutcomeinIA385classinfall2019showsthatmorethan89%students’gradesinthisclassfallwithinthe80%-100%categories.Thatis,uponcompletionoftheclass89%ofthestudentswereabletodemonstratecompetencywith80%accuracyinUnderstandofSQLlanguage;Identify,install,andmanageSQLServer,MySQL,andOracle;UnderstandSQLinjectionandidentifyhowinjectionsareexecuted,andReportPreparation.About10%werebelow80%andnoneofthemdidnotcompletelyattempttheclass.
Inordertohaveacontinuousimprovementthatmayassistthe0-10%students,whodidnotachievethegoalof80%learningoutcomegoal,thefacultyrealizedthatchangesneededtobemade.TheIA385isarequiredcourseforstudentsinundergraduateInformationAssuranceandCyberDefense(IACD)program.Thedepthandscopeofcoverageinthiscoursecouldbechallengingforthosestudentswhowereinthisclasswithoutproperknowledgebackgroundofdatabasesystems.Thestudentsintheclasswerefromverydiversebackgroundwiththeirexpertise,skillsetandknowledgeinthesubjectmatter.Therefore,itisrecommendedtoprovideone-to-onetutoringforthestudentswithoutanypriorbackgroundindatabasesystems.Inaddition,itisrecommendedtoconsiderdesigninganewsectionofthiscoursetointroducefoundationknowledgeofthedatabasesystems.Whilethesechangeswerenotidentifiedfromthepreviousassessmentbecauseanotherclasswasassessed,itisimportantthatthiswasidentifiedforthisclass.ThefacultywilldiscussthesefindingstotheDirectoroftheSISACprogramandtheCurriculumCommitteeforAY22andproposetheysetupatutoringsession,similartowhatwecurrentlyhaveinthecollegeforwritingandmath.Additionally,anadditionalcoursewillberecommendedtoaddressthosestudentswhodonothavethefoundationalknowledgenecessarytosuccessfullyachievetheresultsandSLOsforthiscourse.Whilethesechangescannotbemeasuredastheywerejustidentified,theyareextremelyimportanttotheoverallsuccessoftheprogramandthisparticularclass.TheDirectorwillfollowuponthisandadditionalassessmentsbeconductedoverthenextfewyearsaschangesaremade.Itshouldbenotedthatthetutoringrecommendationhadbeenmadeonthe2019-2020assessment.However,becausechangesasaresultofCOVID-19andthelackofapermanentdirectorforSISAC,nochangesweremadetoaddressthisinAY21.ItwasnotedforAY22.
7
3. Changesmadetostudentlearningoutcomesand/orassessmentprocesses(ifany):
Basedonthefeedbackreceivedfromthefaculty,Directors,andpastassessments,thecommitteemadeseveralchangestothecollegeassessmenttemplate/report/processforAY21:• ‘ClosetheLoop’:Inpastassessments,thefocuswasonassessingthedifferentcourses
eachyearandfailedtoincludefollow-upactions/reviewsfromthepreviousyear’sreport.TheCommitteeaddedthefollowingquestiontothetemplate:PartI:Whatactionsweretakento‘closetheloop’ontheassessmentconductedlastyear?Examplesof‘closingtheloop’weregiventofacultyearlyintheassessmentprocessandtheywerenotedontheassessmentreports.
• Timeline:Inthepast,theassessmentsweresubmittedtooclosetotheendofwintersemester,whichdidnotallowtheDirectorstimetoreviewthereports,recommendchanges,andsubmitthemtotheAssociateDean.Thisyear,anewtimelinewasidentifiedwithboththeAssessmentCommitteemembersandtheSchoolDirectorsthatwouldallowtimeforcompletionoftheassessmentaswellasareviewbytheSchoolDirector.
• PlanofAction:APlanofActionitemwasaddedtothetemplatebasedontheassessmentfindingsforAY20.Thisallowedthefacultymembertoconducttheassessment,showthefindings,andalsoincludetheirplanofactionbasedonthefindings.Becausedifferentfacultymayconducttheassessmenteachyear,thisallowedthemtoseetherecommendationsoftheirpredecessor.Thiswillalsoallowforamoreconsistentandcohesiveassessmentyeartoyear.Becausethiswasthefirstyearitwasaddedtothetemplate,thesuccessorviabilityofimplementingtheseactionswillneedtobeaddressinfutureassessments.Plus,thecommittee,programcoordinators,anddirectorswillneedtoaddresswhowillfollow-upontheactionplans.
• PlanningfortheFuture:Thetemplateaskedthefacultymembertoidentifytheclassesthatwillbeassessedoverthenext5years.Areviewofpreviousassessmentsfoundthatsomeprogramswereassessingthesameclasseveryfewyearsratherthanhavingaplaninplace.Again,becausethiswasthefirstyearthiswasincludedinthetemplate,thiswillneedtobereviewedinfutureassessmentstodetermineiftherolloutofcoursesandSKOswassuccessful.
8
4. Continuousimprovementovertime:
A. Simulation,Animation,andGamingProgram–SAG105L4:Thiscourseisanoverviewofthesoftwareandhardwareassociatedwithsimulationandgaming.Italsoexploresthecreativityanddevelopmentskillsnecessary.Itisanintroductoryoverviewoftheelectronicgamedevelopmentprocessandunderlinesitshistory,contentcreation,strategies,andfuturetrendsintheindustry.SAGstudentstakethiscourseduringtheirfirstsemesteratEMU.TheSLOsforthecourseare:
• InformationLiteracy/Communication• ProblemSolving• CriticalThinking• InterpersonalSkills/Teamwork
ThisyeartheassessmentwasconductedonInformationLiteracy/CommunicationfocusingontheTeamFinalProject.ThisSLOwasalsoassessedinthepreviousyearsandthus,theprogramcoordinatorwasabletocompareresultsforsevenclassesofferedovertwosemesters.Studentswereassessedonthedesignoftheirfinalproject,theirpresentationasagroup,andthebelievabilityandfollow-throughofthesimulation,animation,orgamingthegroupchose.DatawascollectedfortwosectionsoftheWinter2020semesterandfivesectionsforthefall2020semester.Theassessmenttoolwasa1-4LikertScaleonthetenassessmentitems.Theoverallaveragewascalculated,andtheaccumulationofcoursesshowedameanof2.63,whichwasthesameasthe2019/2020assessmentofthecourse(SeeFigure3).
Figure3:SAGAssessmentresultsforSAG105L4TheassessmentshowedthattheInformationLiteracy/Communicationobjectstayedthesameasthepreviousyear.Theprogramcoordinatorfeltthatwasaresultofchangesthey’dmadetotheprogramin2019/2020and2020/2021.Priorassessmentsshowedoutcomeswerevariedandappearedtobebasedontheinstructorsandnotnecessarilybythecriterionmeasured.Becauseofthenumberofsectionsofferedfallandwintersemesters,bothtenuredfaculty,FTLandPTLtaughttheclassesandastandardcourseshellwasnotalwaysusedfortheclass–itdependedontheinstructor.Thus,therewasaconcernthat
9
therewasalackofconsistencyinhowthecoursesweretaughtorassignmentinstructions/expectations.For2019/2020and2020/2021,theyattemptedtohavemoreconsistencyintheinstructorsforthiscourseandfeltthatthisshowedinthecomparisonoftherecentassessment(Fall2020vs.Winter2020).WhilethecourseshaveshownmoreconsistencyinresultsfortheSLOidentifiedbasedontheassessmentsandtheyfeeltheresultsdemonstrateanoverallsuccessatachievingtheSLOintheclass,thefacultyadvisedthattheresultsalsoindicatemoreattentionneedstobefocusedonspecificcriterion,suchasthebasicsofasuccessfuldesign,presentation,believability,production,andcarrying-outdirection.Thefacultywillworkonidentifyingwaystoincorporatethatintotheexistingclassandensurethatitisconsistentacrossallsections.ThefacultyalsorecommendthatstudentscontinuetoseetheEMUWritingCenterstaffmoreoften.WhilestudentswereencouragedtoseekhelpattheWritingCenter,itwasvoluntaryanditwasfeltthattheydidnottakeadvantageofthisasneeded.TheSAGprogramhasreceivedconditionalaccreditationbytheNationalAssociationofSchoolsofArtandDesign(NASAD)withrecommendationsthatSAGisworkingontoincorporateintotheirprogramandfuturecurriculum.Thiswillhelpwithconsistency,assessments,andmeasurements.
B. Unfortunately,theCETtemplatedidnotaskprogramcoordinatorstoincludeexamples
ofcontinuousimprovementovertime.Thetemplatefocusedonclosingtheloopfromthepreviousassessmentandrecommendationsoraplanofactionbasedonthisyear’sassessment.ThiswillbeincludedasarecommendationintheGoalsforAY22.
5. GoalsforAY2021-2022:
ThefollowinggoalswereidentifiedforAY22basedontheoutcomesofpreviousgoalsidentifiedinassessmentreports,aswellasfeedbackfromthefacultyanddirectors.
• StudentLearningOutcomes:Previously,arecommendationhadbeenmadethattheSLOsfornon-accreditedprogramsbereviewed,updated,&/orvalidatedbyfaculty/Directors.ThiswasnotspecificallylistedontheassessmentreportforAY21asanactionitem.
o Goal:ForAY22,thegoalistoincludeinformationontheAssessmenttemplatethatspecificallyasksaboutidentifying/updating/reviewingtheSLOs.
• Participation/Accreditation:SeveralprogramsdidnotsubmitassessmentreportsforAY21.Becauseofthenumberofprogramsinthecollegethatareaccreditedandmustsubmitlengthyanddetailedreportstotheiraccreditingbodies,toaskfacultytosubmitanadditionalassessmentreportischallenging.OneofourDirectorsheadedtheuniversityassessmentprogramforPurdueUniversityandadvisedthatattheiruniversity,accreditedprogramsdidnothavetosubmitanadditionalassessmentreport–theaccreditationreportwassufficient.
o Goal:ThegoalforAY22isforthecollegeAssessmentCommitteetodiscussthisissuewiththeSchoolDirectorstodetermineifamorefeasibleassessmentprocesscouldbeidentifiedfortheaccreditedprograms.
10
• AssessmentTraining:ForAY21,severalfacultymembersconductedassessmentsforthefirsttime,whichresultedinaminimalassessmentreport.Thecommitteealsofoundthatfacultywhocamefromotheruniversitiesconductedassessmentsdifferently(ex:languageandmetricswerenotstandard).
o Goal:Thecommitteerecommendedthatseveralshortinstructionalsessionsbeheldforallfacultyconductingassessmentstonotonlyreview/explainthecollegetemplatetoensureconsistency,buttoalsoinstructfirst-timefacultyontheassessmentprocess.
• Identifying,assessing,andmeasuring‘ContinuousImprovement’:Thetemplatefor2020-2021didnotincludeanythingrelatedtocontinuousimprovementoveraperiodoftime(ratherthantheone-yearassessment).Thus,thefocuswasonthecurrentyear’sassessmentandthen‘closingtheloop’fromlastyear’sassessment.Butprogramswerenotaskedtodemonstratecontinuousimprovementoveralongerperiodoftime.
o Goal:Thecommitteewilllookatincludingthisinthetemplateforthisnextyear’sassessments.
• Assessingandmeasuringchanges:Asthecollegecontinuestoimproveitsassessmentprocessandamendtheassessmenttemplate,itwasevidentthisyearthattheassessmentprocessfailedtofollow-upandassessthesechangestodeterminewhetherornottheyimprovedtheSLOsthatweremeasured.Theassessmentwasconducted,theassessmentmade,andrecommendationsofferedtoimprovethecourse/SLOs.Insomecases,changesweremade,courseswereadded,orcontentwaschanged;however,wedidnotgobackandmeasurethosechanges.
o Goal:Thecommitteewillneedtodiscusshowtoaddthisintheassessmentprocess.Itisdifficultbecausesomecoursesareonlyofferedonceayearoreveryotheryear.TheCommitteewillneedtoaddressthiswiththedirectorsandprogrammanagers.