Stepping Stones: Capacity building in engineering education

Session S1H

Stepping Stones: Capacity Building in EngineeringEducation

Arnold N. Pears1, Sally Fincher2, Robin Adams3 and Mats Daniels1

Abstract— CeTUSS (www.CeTUSS.se) is an engineering edu-cation center established by the Swedish Council for Renewalof Higher Education in 2004. During 2006/2007 CeTUSS funded”Stepping Stones”, a multi-phase (project based) initiative fortertiary engineering educators at Swedish Universities. The aimwas to build a community of engineering educators and to in-crease their familiarity with evaluation and research approachesto assessing the impact of classroom interventions.

Stepping Stones was based on the earlier US, UK andAustralian initiatives; the Scaffolding, Bootstrapping and BRACEprogrammes. The approach uses a joint, multi-method, researchstudy to raise awareness of relevant theory, while simultane-ously supporting community development. Community buildingis achieved through joint work and shared experiences whichpromote convergence on a common set of values and ideals inrelation to scholarship of teaching and learning. Investigative”capacity” was enhanced by drawing together a Swedish pool ofengineering education expertise.

Stepping Stones consisted of three phases. The first phasewas a week long workshop examining relevant theory andempirical study design in engineering education research. Thisworkshop introduced an ”experiment kit”, a protocol detailingexperimental design of the project that participants jointlyimplemented in phase two. During phase two the participantsgathered data in their own classrooms, contributing to a jointcorpus of material for analysis in phase three. During thedata collection process participants administered and validated avariety of instruments; surveys and interviews (including photoelicitation), and concept map collection using Explanograms (atool for automating collection of handwritten data developedby the CeTUSS center). The final phase was a week-longworkshop where participants analyzed the aggregated data andproduced a written report, ’What is the Word for ”Engineering”in Swedish: Swedish Students Conceptions of their Discipline’(http://www.it.uu.se/research/publications/reports/2007-018).

INTRODUCTION

Enabling collaboration and supporting and enhancing col-laborative development of educational competence is a keyaspect of dissemination and capacity building initiatives. Oneaspect of capacity building is assisting people in locating andcommunicating with one another in order to collaborate andbuild on prior knowledge. Stepping Stones utilizes a provenproject based model for community/capacity development aspart of a larger initiative to promote diversity and movetowards a more scholary and informed approach to teachingpractices in Engineering.

In the work described here we target two aspects of capacitybuilding for Swedish Engineering Education. How do we com-

1IT Department, Uppsala University, Box 337, 751 05 Uppsala, SWEDEN,[email protected]

2CS Department, University of Kent, Canterbury, UK3Department of Engineering Education, Purdue University, IN, USA

Fig. 1. Applications to study in Engineering Programmes

bine and build upon national engineering education researchcompetence, reinforcing and diversifying the network of activeacademics working in Engineering Education in Sweden? Canwe collect broad ranging data about views of Engineering thatprovide insight useful when discussing the Swedish studentrecruitment and retention issue?

The remainder of the paper is structured as follows. Inthe next section we discuss the motivations for this work,and identify some related theoretical and study results. Theimplications and uses of the study data for engineering teach-ing at University level are then discussed and a number ofinvestigative directions are described. We conclude with someobservations about the possible uses of the data and outlinefuture work on the development of recruitment and retentionstrategies for Engineering Education in Sweden.

BACKGROUND

Engineering education in Sweden, as in the rest of the world,is experiencing a decline in student interest (see Figure 1).There are concerns about the ways in which students thinkabout engineering education, why they join an academic pro-gramme in engineering, and why they persist in their studies.In this context one of the aims of CeTUSS is to investigatethe Swedish student experience and to identify and supporta continuing programme of research leading to changes inhigher education for engineers. Supporting this long termgoal demands initiatives to expand the National capacity fordisciplinary pedagogic investigation.

Stepping Stones addresses these concerns in a multi-researcher, multi-institutional study that spans eight Swedish

978-1-4244-1970-8/08/$25.00 c©2008 IEEE38th ASEE/IEEE Frontiers in Education Conference

S1H-11

October 22–25, 2008, Saratoga Springs, NY

Session S1H

Fig. 2. Overview of Sepping Stones studies

universities and a range of engineering programmes. As thestudy is situated in a uniquely Swedish education setting, itpermits us to explore ”a Swedish perspective” on conceptionsof engineering. Stepping Stones was based on a model ofresearch capacity-building previously instantiated in the USAand Australia [1].

RELATED WORK

Stepping Stones draws on a variety of literature to define itsresearch context and focus. Persistence in tertiary educationhas been investigated in a number of areas. Perceptionsamong school students influence their choice of career pathand motivation to study engineering [2]. Students’ universityexperiences in Science, Technology and Mathematics havealso been investigate in relation to academic persistance [3]–[6].

Evidence of a disconnection between engineering practiceand student experiences of tertiary engineering educationhas been advanced by a number of researchers. Studies ofpracticing engineers show a shift from a purely technical focustowards an increasing appreciation of social and environmentalfactors. See also Jonassen et al. [7] and Lethbridge [8] fordiscussions of the current dielectic between education andpractice in engineering. Issues related to ancillary skills whichmight be acquired ”accidentally” are discussed by Walther andRadcliffe and Berggren et al. [9], [10]

There is a small but growing body of literature on col-lege students’ understanding of engineering and engineeringpractice. For example, ethnographic investigations into collegestudents’ perceptions of engineering reveal the predominanceof technical knowledge and mathematical problem solvingwhich may be related to students’ limited experience withformulating and defining complex and ambiguous problems[11]. Turns et al. [12] used a word association task to studygraduating civil engineering students’ schemas of civil en-gineering and found that technical knowledge predominatedsignificantly over such issues as communication, multidisci-plinary teams, and global and societal context issues. Downeyand Lucena [13] also examined how students involved in de-sign experiences perceive the distinction between science anddesign. Their findings suggest that students perceive designas a lesser subset of the engineering method of mathematicalproblem solving, and as such are ill prepared for dealing with

Fig. 3. Sepping Stones data demographics by institution.

the ambiguities and subjectivity inherent in engineering designproblems. Similarly, Newstetter and McCracken (2001) foundthat freshmen engineering undergraduates’ early conceptionsof design tend to conceptualize design as an artistic, creativeprocess - ”a blaze of creative light that strikes some andnot others” (pg. 70). In a related study, Mosborg et al.(2005) found that advanced practicing engineers ranked asmost important among a list of 23 design activities ”problemformulation” and ”communication” ”Building” was rankedamong the least important activities and ”creativity” wasincluded in neither the highest nor lowest ranked groups ofterms.

Our study drew on this work when formulating the focalquestions for the research. Many of these questions evolvedas we delved deeper into the dataset. These changes aredocumented in the individual studies we overview later in thepaper.

DATA COLLECTION

Stepping Stones data collection is based on five centraldesign decisions. Firstly, we did not want to impose anexisting framework of conceptions of engineering on studyparticipants. Using elicitation techniques; such as interviewsand concept maps seeks to reveal participant’s underlyingconceptions of engineering. We sought to allow importantaspects of the experience of engineering education to emergeduring the investigation. Secondly, we wanted the study tobe domain independent so that comparisons could be madeacross engineering domains. This was done by using general”engineering” terms in the study instruments rather than termsthat are specific to a sub-discipline (e.g., mechanical engi-neering or computing). Given the complexity inherent in ourresearch questions we wanted triangulation, this is achieved bya data collection approach designed to increase the likelihoodof contradiction or corroboration of other study findings. Thiswas achieved by combining several different approaches andcollecting both qualitative and quantitative data. Finally, wewanted the study to be of sufficient scale to allow general-isation. Simultaneously we wanted to leverage the power ofexisting studies reporting results from validated data collectioninstruments. These last objectives were achieved using a web-


S1H-12


Session S1H

Fig. 4. Frequency of student/instructor interaction distributed by Institution

based survey [14], [15] which collected data from manyparticipants at many institutions in Sweden. The completestudy comprises four data collection instruments: a web-basedsurvey, the construction of a concept map, a critical incidentinterview and a photo elicitation interview.

Data was gathered over the academic year 2006/07 from tenSwedish institutions (although not all institutions gathered allforms of data). For the concept map task and interviews, datawere given a unique identifier of the form: AF01 where thefirst letter represents a unique institution code, the second letterrepresent experience level, of the form F (First Year Student01), G (Graduating student), A (Alumni) or E (Educator) andthe number represents a unique participant. Participants mayhave contributed to all parts of the study, to just the survey orthe just the concept map task and interviews.

STUDY OUTCOMES

This section summarises the preliminary results for thefive preliminary studies described in Figure 2 (Study E ispresented last, as it builds on some of the earlier studies). Asis apparent from the table some studies focus on a single datasource, while others combine and compare information fromseveral data sources. The unit of analysis varies in the differentstudies - some focus on institutional level characteristics, someon participant level characteristics, and some focus on bothinstitutional and participant level characteristics.

Study A

In this part of the data analysis, the validity and implicationsof the constructs 1 of the survey data were compiled, plottedand analyzed per institution to explore students’ perceptionsabout engineering education at their respective university.Overall the results from this part of the data analysis reveallittle that is surprising. In other words, there were few in-stances of noticeable differences across the institutions. Withthe exception of Construct 12 (”Frequency of interaction withinstructors” (see Figure 4), there are few constructs wherethere is an observable difference between the universitiessurveyed. This is also true for constructs 11c, 13a and 13b.

1See the Stepping Stones technical report, Appendix G

Fig. 5. Common terms in Concept Maps

Therefore, it seems that institution is not an interesting unit ofanalysis. Instead, it suggests that this data, or the constructsthereof, may be more usefully analyzed with respect to otherfactors, such as gender or disciplinary program.

Another implication of this analysis is that it suggests thatthere is considerable consistency in what Swedish engineeringstudents think about engineering education at their universities.In this respect there appear to be no significant differencesamong the subjects of the survey sample for most of theconstructs. The only case, Construct 12, which characterizesthe frequency of interaction between students and instructors,is hardly surprising due to such institutionally variable factorssuch as the availability of resources, pedagogical approach,and the number of students.

To facilitate subsequent analyses, we identified a set ofcategories of engineering education disciplines based on theeducational programs that occur in the survey sample. Wepropose that this should be consistently applied in all areasof the data where educational programs manifest themselves2.

Study B

This study draws on three of the Stepping Stones datasources; namely the web based questionnaire, the conceptmaps (CM) recorded using explanogram technology, and theinterviews. The primary goal of the study was to exploreconcept map data, trying to find interesting patterns that couldbe further analyzed. A secondary goal has been to exploredifferent approaches to analyzing concept maps. Our overar-ching research question is: What can we learn about students’and educators’ views of engineering from their Concept Maps(CMs); how do they relate engineering to other concepts andwhich seem to be the most and least important concepts intheir CMs? To operationalize this question we developed a setof more specific questions:

• Which are the most or least important concepts in thedrawings?

• Can the structure or appearance of the drawings becategorized in some way?

2The table is available in the Stepping Stones technical report, page 13


S1H-13


Session S1H

Fig. 6. Response Categorisation Q1

• Are particular concepts ”closer” to engineering than oth-ers?

• Are there any differences for different groups of subjects?One contribution of this analysis is a classification of which

concepts were central in the drawings. We defined ”central”with reference to the following properties: central in a spatialsense (in the ”middle” on the top of the drawing serving as aheader, having many links, drawn as frame that includes otherconcepts, or as explained by additional text. Several conceptscan simultaneously be considered as central. Which of theconcepts appeared as central most often or rarely? To find thecentral concepts three researchers visually inspected all CMsand categorized them until they reached full agreement. AllCMs with engineering as one of the central concepts weresorted out in a first pass. We then inspected the remainingCMs a second time and noted down all central concepts. OneCM did not have any concept that could be interpreted ascentral; it was drawn as a sequence of concepts in a singleline. The central concepts are summarised in Figure 5

Study C

One of the focal questions of the Stepping Stones projectdeals with conceptions of engineering in Sweden. We wantedto know what students and educators think are the elements ofengineering. We focused on three of the interview questionswhich ask specifically about engineering and what engineering”is”. Questions 1 and 2 are at the beginning of the 30 minuteinterview, while Question 5 occurs right at the end of theinterview.

Q1: In a few words, what would you say realengineering is?Q2: Can you give me some examples of engineeringin the world?Q5: After everything we’ve talked about, what wouldyou say ”engineering” is for you?

All responses to these questions were digitally clipped fromthe transcripts and printed out. Keywords and phrases inQ1 were first marked and then grouped into ten differentcategories by one person (see Figure 6). Another personverified the categories. The same process was used for Q5

Fig. 7. Response Categorisation Q2

where we found that the same categories could be used togroup responses. The responses to Q2 were also treated in thesame way, but since these were mostly nouns that left littleroom for interpretation only one person coded these. For Q2we identified 11 different categories (see Figure 7).

There are two noticeable changes in the way the participantscharacterize engineering between Q1 and Q5. When answeringQ1 at the beginning of the interview 17% of participantsdescribe engineering using examples of different academicsubjects (e.g. mathematics, physics). This should be comparedwith the comments made to Q5 towards the end of theinterview and critical incident exercise, where that numberhad decreased to 8%. We also note that the proportion ofparticipants that mentioned the impact of engineering onsociety increased from 12% to 28% between questions 1 and5 during the interview.

Study D

Study D uses the data from the photo elicitation interviews(where participants were asked about the associations ofengineering they had with three different images). There wasalso a final interview question regarding how the participant’sideas about engineering might have changed.

During the interview the participants were asked ”Whatassociations of engineering does image A (B, C) have foryou?” At the same time the corresponding photo was shownto the subject. The section of the interview dealing with photoelicitation has been extracted and analysed for specific wordsand concepts. The associations have been classified using acontent based coding scheme . Only Image A has currentlybeen analyzed (the relevant image is shown in Figure 8). Weintend to analyse the Data for the other pictures in a futurestudy.

Six classes of association are identified by our analysis.

Plan :Participants associate the image to planning a project,analysing something, interpretation of results, devel-oping something new or solving a problem. Grad-uating students seemed to make this association to


S1H-14


Session S1H

Fig. 8. Frequency of student/instructor interaction distributed by Institution

a higher extent then first year students; however thedifference was not significant. Also 40% of the maleparticipants had this association compared to 13% ofthe female participants.

Male :Participants that observe and associate that there areonly men in this picture are classified into this group.Female participants made this association to higherdegree. It is also worth noting that educators have thisassociation. 50% of females and 50% of all educatorsmade this association compared to 7% amongst allmale participants.

Team :Participants that associate teamwork and peopleworking together with this image are classified intothis group. The only noticeable difference is thateducators are slightly more represented.

Sci :Participants that associate picture A with mathemat-ics, physics or scientific research fell into this group.In this case there is no noticeable difference.

Eco :Participants that associate the image with economics,the stock market, project economy or statistics areclassified into this group. First year students andmale participants are more represented. 36% of thefirst year students made this association compared to23% of graduating students. 30% of male participantsmade this association compared to 6% of the females.Many participants with this association said the pic-ture could equally well represent economists anddidn’t have anything at all to do with engineering.

Old :Many participants associated this picture with ”oldways” of engineering or made some comment that itwas old, maybe from the fifties. Graduating studentsand educators are represented to higher degree in thisgroup. 44% of the educators made this associationcompared to 19% of first year students and 27%

graduating students. Also 44% of female participantscompared to 24% of male participants made thisassociation.

Study E

Study E examines the data collected from survey partic-ipants looking for gender differences. Such differences arealso checked by triangulation with other data we collectedin the project. Of 521 survey participants, 108 were females,383 males and 30 did not state their gender. The study firstlooks for obvious gender differences in the answers given bythe females as compared to the males, and then if differencescould also be seen in the concept maps and the results from theinterviews and the concept map debriefs. We chose to focuson a few aspects, such as why do students choose to studyengineering, how do they rate their own skills as compared totheir classmates and what traits do they believe are importantfor a working engineer.

In question ten, the students were asked to rank ten differentstatements about why they chose to study engineering. Thealternatives contained statements like ”Technology plays animportant role in society”, ”Engineers make more money thanmost other professionals”, and ”My parents want me to be anengineer”. For the complete list of statements see the SteppingStones Technical Report, Appendix A.

Options for each question were; ”not a reason” ”minimalreason” ”moderate reason” and ”major reason” There were nosystematic differences between the male and female responsesexcept in the two statements ”My parents want me to be anengineer” and ”An engineering degree will guarantee me a jobwhen I graduate”3. The males did not identify the influence oftheir parents as a strong motivation for choosing engineering,while the females did. The females indicated, to a larger extentthan the males, an expectation that an engineering degreewould guarantee them a job after graduation.

With regard to perceptions of professional practice, therewere no differences attributable to gender. In question 13of the survey, participants were asked to rate how importantthey considered different skills and abilities for a becoming asuccessful engineer. The rating was ’not important’, ’somewhatimportant’, ’very important’ and ’crucial’. There no observabledifferences in the female and male answers, other than a largerspread in the male answers regarding ’self confidence’ and’communication skills’. Males rated communication skills asslightly more important than females.

CONCLUSIONS AND FUTURE WORK

The Stepping Stones project has gathered a rich data setfor exploring conceptions of engineering from a Swedishperspective. Initial analyses of a variety of factors has beensummarised here in studies A-E reporting on experiences ofengineering study and their impact on decisions to continuteto study engineering at Swedish Universities, as well asperceptions of which concepts/tasks are central to engineering.

3These results are shown in figure 13 and 14 of the Stepping Stones report,respectively.


S1H-15


Session S1HWe conclude that the experience of engineering studies isquite uniform in the Swedish education system. The mannerin which students conceive engineering, and the elements ofengineering practice that they perceive as important, are worthconsidering in an international context and contrasting withstudies in other countries. This may help to identify areaswhere perceptions of Swedish engineering are different tothose in other countries and educational systems. Finally wehave considered gender differences in relating to engineeringand the perception of engineering as a ”male” discipline. Thisaspect of the data deserves further investigation given thelow representation of women in swedish tertiary engineeringeducation. In terms of continuing work we intend to undertakethe following analyses:

Surveys:An analysis by discipline. Concept maps: Analysinghow the concept maps develop over time, usingthe explanogram representations. For example, ex-amining the sequences graphically and reformulatingthem as stories or narratives that could be furtheranalyzed. For example, could we then tell the cat-egories’ different stories? Some of the participantsforgot or excluded certain words. Is there a patternto this or is it just neglect to check that all conceptswere used? How can we relate this to the debriefinformation? Especially interesting would be relatingthese sequences with questions one and three fromthe concept map debrief, that ask students to relatethe terms used in the mapping task to their universitycourses.

Interview and survey data combined:An analysis of question 38 of the survey with respectto questions 1, 2 and 5 in the interview. For example,it should be possible to create a list of terms studentsassociate with engineering and then compare thesewith the responses in the interview. Also, at thisstage, no aggregation of the terms from the surveyhas been performed. Early analyses of the survey,reported here, indicate that ”problem solving” and”mathematics” are terms that participants frequentlyuse for describing engineering. In addition, it wouldbe useful to continue the analysis of the photoelicitation interviews.

Gender:What perceptions can be identified related to the”gendered-ness” of engineering in the Swedish con-text? How might this affect our ability to recruitand retain women in engineering programmes inSweden, and what similarities and differences arethere compared to other major studies?

ACKNOWLEDGEMENTS

The Stepping Stones project was funded by Radet for hogreutbildning (the Swedish Council for Renewal of Higher Edu-cation) and by Myndigheten for natverk och samarbete inom

hogre utbildning (the Swedish Agency for Networks and Co-operation in Higher Education). It was run within the contextof the Nationellt amnesdidaktiskt Centrum fr Teknikutbildningi Studenternas Sammanhang project (CeTUSS), a nationalcentre for pedagogical development in technology education ina societal and student oriented context. Any opinions, findingsand conclusions or recommendations expressed in this materialare those of the authors and do not necessarily reflect the viewsof any of these supporting bodies.

We thank the Stepping Stones participants, Jonas Boust-edt, Jrgen Brstler, Peter Dalenius, Gunilla Eken, Tim Heyer,Andreas Jakobsson, Vanja Lindberg, Bengt Molin, Jan ErikMostrm, and Mattias Wiggberg,for their contributions to thisproject.

REFERENCES

[1] S. Fincher and J. Tenenberg, “Using theory to inform capacity-building:Bootstrapping communities of practice in computer science educationresearch,” Journal of Engineering Education, vol. 95, no. 4, pp. 265–278, 2006.

[2] E. W. E. C. (EWEC), “Extraordinary womenengineers final report,” 2005. [Online]. Available:http://www.engineeringwomen.org/pdf/EWEPFinal.pdf

[3] E. Seymour and N. Hewitt, Talking about Leaving. Westview Press,1997.

[4] M. Besterfield-Sacre, C. Atman, and L. Shuman, “Engineering studentattitudes assessment,” Journal of Engineering Education, vol. 87, no. 2,pp. 133–141, April 1998.

[5] M. Besterfield-Sacre, M. Moreno, , L. Shuman, and C. Atman, “Genderand ethnicity differences in freshmen engineering student attitudes: Across- institutional study,” Journal of Engineering Education, vol. 90,no. 4, pp. 477–489, October 2001.

[6] M. Hutchinson, D. Follman, and M. Sumpter, “Factors influencingthe self-efficacy beliefs of first-year engineering students,” Journal ofEngineering Education, vol. 95, no. 1, pp. 39–47, 2006.

[7] D. Jonassen, J. Strobel, and C. B. Lee, “Everyday problem solving inengineering: Lessons for engineering education,” Journal of EngineeringEducation, vol. 95, pp. 139–151, April 2006.

[8] T. Lethbridge, “What knowledge is important to software professionals?”IEEE Computer, vol. 33, no. 5, pp. 44–50, 2000.

[9] J. Walther and D. Radcliffe, “Engineering education: Targeted learningoutcomes or accidental competencies?” in Proceedings of ASEE Con-ference, June, Chicago, 2006.

[10] K.-F. Beggeren, D. Brodeur, E. Crawley, I. Ingemarsson, W. Litant,J. Malmqvist, and S. Ustlund, “CDIO: An international initiative forreforming engineering education,” World Transactions on Engineeringand Technical Education, vol. 2, no. 1, 2003.

[11] G. Downey and J. Lucena, “Engineering selves: Hiring into a con-tested field of education,” G.L. Downey and J. Dumit (eds), Cyborgsand Citadels: Anthropological Interventions in Emerging Sciences andTechnologies, pp. 117–142, 1997.

[12] J. Turns, J. Temple, and C. J. Atman, “Students conceptions of theirengineering discipline: A word association study,” in Proceedings ofASEE conference, 2000.

[13] G. Downey and J. Lucena, “When students resist: Ethnography ofa senior design experience in engineering education,” InternationalJournal of Engineering Education, vol. 19, no. 1, pp. 168–176, 2003.

[14] O. Eris, H. Chen, T. Bailey, K. Engerman, H. Loshbaugh, A. Griffin,G. Lichenstein, and Cole, “A development of the persistence in engi-neering (PIE) survey instrument,” in Proceedings of ASEE conference,2005.

[15] ——, “A preliminary analysis of correlates of engineering persistence:Results form a longitudinal study,” in Proceedings of ASEE conference,Oahu, Hawaii, 2007.


S1H-16


Stepping Stones: Capacity building in engineering education

Documents

Transcript of Stepping Stones: Capacity building in engineering education