The development of a working Undergraduate

Research Project - Lessons from the IP Challenge

A.L. Nel, W.A.Clarke Member IEEE and N. Clarke

Department of Electrical and Electronic Engineering Science,

University of Johannesburg,

P.O.Box 524, Auckland Park, 2006, South Africa,

Email: andren@uj.ac.za, willemc@uj.ac.za, norahclarke@gmail.com

Abstract—In this paper we describe the development of a unique

undergraduate engineering research program that has been run

for two years. It has focussed on the proposition that

undergraduate students if properly trained and adequately

mentored can produce publishable results while still at the

undergraduate level. The aims, curriculum development and

preliminary results of students that have attended the course are

discussed. Proposals for changes during the 2011/2012 IP

Challenge (the marketing name for the Image Processing

Research Training Course) are also discussed.

Keywords- undergraduate research; undergraduate research

curriculum; managing research

I. INTRODUCTION

Undergraduate research is almost by definition nothing new in engineering education. Engineering degrees worldwide have had two final year capstone courses for at least the past 40 years. The one capstone course is aimed at engineering analysis and synthesis – the final design course. This design course is aimed at developing the student’s ability to specify, design, analyze and implement an engineering product or project. The second, and often neglected, capstone course, the so-called research project, is aimed at the investigation of an engineering problem –most often from a scientific viewpoint. Such a course was already introduced at MIT in the undergraduate program in 1969 [1].

It was therefore surprising that as long term academics in an engineering faculty that we should feel that our own undergraduate research courses did not seem to perform as expected. The IP Challenge was thus aimed at developing a much more realistic research experience for undergraduate students. The aim of this paper is to document the reasons for developing an undergraduate research which differs substantially from those that are presented in most curricula. We will report on the design, time involvement, outputs and feedback received from both staff and participants.

The further development of these types of undergraduate courses aimed at developing a research and/or investigative perspective blossomed during the late seventies and early eighties. By the turn of the millennium there were scarcely any degree programs that did not include such a course, and international recognition and accreditation of engineering programs was often contingent on such components [2].

Even though it would appear that there is broad consensus about the desirability of such exposure to undergraduate

research, [3] reports that at a meeting arranged by the National Science Foundation in Washington DC two questions were raised:

What are the essential features of a successful undergraduate research experience?

What are the benefits of a successful undergraduate research experience?

From these two questions there is a clear indication that there is some confusion about what the aims of these types of course are and yet mentors and faculty are almost unanimous in their recognition of the benefits of such undergraduate research programs [4]. Lopatto [3] for instance reports the most often cited perceived benefits of such an undergraduate research program by staff and students as shown in Table I.

TABLE I. PERCEIVED BENEFITS OF AN UNDERGRDUATE RESEARCH

EXPERIENCE

Benefit Perceived benefits (reworked from [3])

Staff perspective Student perspective

1 Learn a topic in depth Enhancement of professional credentials

2 Apply knowledge to a

real situation Clarification of a career path

3

Learn to use

appropriate

methodology

Understanding the research process

4 Learning to work

independently Learning a topic in depth

5 Learn to design

solutions to problems

Develop a continuing relationship with faculty

6 Improve oral

presentation skills

Learning to work independently

7 Improve written

presentation skills Learn laboratory techniques

8 Appreciate what

scientists do

Learn tolerance for obstacles

9

Develop an

orientation to future

work

Understand how scientists think

10 Learn to use scientific

literature

Understand how professionals work on real problems

What is clear is that neither the staff nor the students are in fact listing benefits that could not accrue from any other type of

course in the final year engineering curriculum. For instance, a final year capstone design course could as easily be constructed to address virtually all of the perceived benefits listed above.

In a recent report Healey and Jenkins [5] state “For us the key to developing undergraduate research and inquiry is to mainstream it and integrate it into the curriculum for all students.” They have in [5] put forward a model of the interaction of teaching and research in a normal undergraduate curriculum where the four models of teaching – research interaction can be categorized as:

1. research-led: learning about current research in the

discipline

2. research-oriented: developing research skills and

techniques

3. research-based: undertaking research and inquiry

4. research-tutored: engaging in research discussions

While this is laudable they also acknowledge that “We

would not want, however, students to spend nearly all their time in the top half (categories 1 and 2 above), as tends to happen in some problem-based learning courses.” What appears from their report however contrasts with our own expectation. Much of the reported publication of the lower undergraduate research projects which are curriculated and used as case studies appear in “either e-journals or undergraduate research journals” [4].

While it is therefore not unique to talk about research in the undergraduate engineering education and with the ubiquitous presentation of such courses that there is still some uncertainty regarding the benefits and the depth achieved in actual courses, which is surprising [5]. This is clear from a number of sources that question the actual research component in so-called undergraduate research experiences/courses. As an example in [5] we see that “Over the next six weeks, students register into one of ten clusters (of up to 45 pairs) based around biological themes. With the support of online tutors (PhD students), they write and upload a paper (15%) and construct a PowerPoint presentation (10%) in pairs.” While we would not dispute that this has its merits it is doubtful whether “online tutors” or “construct a PowerPoint presentation” will have the same impact as we planned for with our proposed process.

Extensive reviews of research program outputs at our university and a short review of other Southern African university project reports, has lead the authors to the conclusion that it is not the uniqueness of the idea of undergraduate research, but rather a change in emphasis that should be the present concern.

It would appear that what was (and presently often still is) lacking is a structured introduction to the research process at the undergraduate level. It would also appear that in general many students seem to feel that they are simply “gophers” in the research laboratory rather than real researchers [6]. Interestingly Hunter, Laursen and Seymour note that “For instance, bottle washing may be a necessary part of laboratory research, but if a student is left to wash bottles every day, it is unlikely that he or she will make the same gains as a student

who is actively engaged and making decisions1 about how best to move the research forward.” [7].

The limited course credits result in a limited number of hours in these undergraduate courses in which to instill real research tools and techniques – and the opportunity to use these tools in the prosecution of a real research project.

II. RESEARCH DESIGN AND METHODOLOGY

The research design that has guided this study was qualitative, exploratory, descriptive and contextual [8]. A functional approach to the actual research was followed, which means that the research was aimed at improving the particular practice (of research training) – here the related ethical practices are those of training young engineering students in research methods via directed lectures and unguided / open ended practical problems of a direct research nature within an educational context [9]. The research was conducted on the basis of the scientific principles of logic and justification as described later in Section III where the research process is described [9]. An important aspect of the principle of justification involves establishing trustworthiness by applying the strategies for the achievement of rigor in qualitative research, as based on Lincoln and Guba’s [10] criteria for trustworthiness in qualitative research. Of the criteria for establishing trustworthiness – credibility, transferability, dependability and confirmability – credibility was most significant in the research process in question.

Credibility is related to establishing how confident the researchers are with the truth of the findings, within the context of the study [11]. This was achieved through the description of experiences as they were lived by the research participants while taking part in the IP Challenge. Krefting [11] suggests some strategies to ensure credibility, of which triangulation was a key aspect in this study. Triangulation of data sources involved maximizing the range of data sources that might contribute to the researchers’ understanding of the way that participants had experienced their involvement in the project.

The research took the form of a contextual, specific and descriptive case study of the educational benefits experienced during the course of the IP Challenge. Case study research is an in-depth qualitative investigation of a bounded system or current phenomenon by means of detailed data collection and the utilization of multiple sources of information, such as the diversity of interviewees who participated in this study. The result is a case description and case-based themes concerned with generating knowledge of the particular [10, 12, 13].

A random sample was drawn from the population of participants involved in the project. This sample consisted of a number of the participants, staff and mentors from whom the researchers were likely to learn the most, given that there were only limited opportunities to do the interviews without disturbing the research design itself, [13, 14].

Data collection took place through a number of conversational, semi-structured interviews with the selected

1Our emphasis

participants, with the aim of eliciting accounts of the interviewees’ experiences during the IP Challenge. During this phase of the research process we were obliged to consider our own ethical responsibilities as researchers. Interviewing as a process of human interaction may pose certain risks to interviewees, such as potential embarrassment, violation of privacy, misunderstandings, and conflicts in opinions and values. As researchers we had to remain mindful of the duty to balance our need to gather valid data with the participants’ rights to privacy and autonomy [14].

A number of ethical measures related to informed consent were taken to prevent the potential risks implicit to the research interview situation. Participants were invited to take part in the research voluntarily and were told why they were singled out for participation. Background information about the study was provided. Participants were informed of the competence and background of the researcher who conducted the interviews. Information obtained during the data collection process remained confidential, unless otherwise agreed upon in advance. Participants were also informed of their right to withdraw from the research without penalty. Other general ethical principles were adhered to during the research process, such as that the research should not bring harm to participants and that participants’ right of privacy should be respected.

Figure 1. Design of the IP Challenge as an experiment

The empirical data in this study consisted of the papers produced by the participants, video recordings of the interviews and subsequent transcriptions of the interviews, along with field notes made by the interviewer during the research period. The process of data analysis consisted of the coding of the empirical data, with the emphasis on interpreting for understanding. The texts and video footage were revisited as needed in order to continue the process of identifying the identified benefits.

III. DESIGN OF AN UNDERGRADUATE RESEARCH PROJECT

At the outset the authors (full time staff at the University of Johannesburg (UJ)) were concerned that the outputs being achieved by the typical undergraduate research course student at UJ were not in line with our expectations as researchers. It should however be noted that our students’ performance was

not out of sync with the outputs from engineering faculties at other South African universities.

Thus prior to changing the present courses we applied our normal research process to the problem of undergraduate research experience for a student. The result is shown in Fig. 1. The figure makes clear that we developed the course so that it would itself form a research project of reasonable scope with the deliberate aim to improve our understanding of the parameters that control the undergraduate research process.

A. The idealized undergraduate research course

Taking into account the fact that the items from Table I (reworked from [3]) somehow do not reflect our expectations of a research course, we set forth an agenda for what we considered the ideal contents and processes that would constitute a future course.

This agenda can be summarized by Fig. 2. In the figure it is clear that a systematic review of the research process (based on a simplified scientific paradigm model) leads to a consideration of at least 5 elements, viz. contents, process, inputs, outputs and agents.

Figure 2. The IP Challenge as system

B. The idealized inputs

We commenced with a review of what we considered the set of inputs that would be most enabling for young engineering students to perform research, at least at the level of inexperienced but more senior research students. The final list of curriculum requirements is given in Table II. That some of the items listed in Table I also occur in Table II is simply a result of the nature of research as a communicative process.

As a first attempt we were then confronted with the need to develop materials which were accessible to undergraduates but still conveyed the essentials of each of these. Thus were born special courses on article writing and research report reviewing as discussed in [15].

We began with an aim to develop that specific set of skills, techniques and tools which are common to the vast majority of engineering researchers (without determining this by research) techniques and tools were to some extent driven by the

questions raised in [16], [17] and [18]. The specific content that was thus included was eclectic and driven by our own experiences as researchers who were always in an emerging country university where research support and supervision is at a premium. Furthermore we always understood that during the period in which the IP Challenge was to take place the participants should be freed from other study commitments. The only way to make this possible was to present the course during the long year end break when there are no other academic courses. The content finally selected can be summarized as:

TABLE II. IP CHALLENGE COURSEWORK INPUTS

Course Formal inputs

Content Presentation

format

Time

(hours)

1 The scientific paradigm

and research process

Lecture and

practical 8

2 Reading for research Lecture and

practical 12

3

Research methodology –

hypotheses and experimental development

Lecture and

practical 12

4 Introductory image

processing

Lecture and

practical 16

5 GPU programming Lecture and workshop

16

6 Article writing and

critiquing

Guided writing

workshop 8

Developing the excavation based research skills in the

participants. These skills included the ability to find

sources of information, exploring various data fields,

identifying material of use for the present need.

Ensuring that the participants reached a proficiency

level in a narrowly focused area that would almost

guarantee paper acceptance at a peer reviewed

conference level.

Ensuring that the participants are mentored by both

graduate students as well as research staff. Instilling a

love for and desire to excel at research is often best

transmitted in a mentor – mentee relationship.

Ensuring that the participants develop an

understanding of the research process that is most

often the underlying process in academic laboratories.

Making the exercise so realistic that significant

numbers of the participating students would realize

that incredible moment of being possibly the first

person to ever have contemplated the results of the

research project. Even the students who do not reach

this level must be surrounded by ones that do –

research excitement is highly contagious and students

exposed to it are unlikely to not be affected.

Using an academic environment in which research can

be conducted without expensive equipment or the

need for time-consuming experimental setup

developments. Two of the authors had experience in

the broad image processing research area and this

became the area in which the students would start.

C. The idealized process

We held from the outset as important three ideals for the process. The first is that direct competition would inspire the participants to excel as young researchers. The IP Challenge was therefore designed to be a challenge in two senses – firstly to achieve the level required for a publication, but secondly to “win” – implemented in our case by offering the two best participants a one year paid research assistantship within the image processing laboratory2.

The second ideal from a process perspective was that simply dropping the students into an undergraduate research course is suboptimal – although it is often the way that these undergraduate research courses are managed. We felt that if we were to ensure that the students were to advance more rapidly than normal, we would have to structure and manage the process much better. The various processes we used are:

Research process training – the scientific paradigm in

practice. Done via small thought experiments in a

lecture and explicated via quick Matlab example

training. (“Research” questions were simple tests of

hypotheses that students could not prove

mathematically but could easily develop numerical

solutions to.)

Research methodology training – explaining the

differences between theory, quantitative experimental

research and qualitative based research in practice.

The practical exercises were implemented as critical

guided readings of a number of poor and excellent

research articles in journals.

Research related reading and writing skills

development – ensuring that introductory researchers

can read accurately and critically and then write in a

similar fashion. Done via a guided reading and

writing workshop format [15].

Basic prototype implementation skills via Matlab

and/or Mathematica – rapid introduction to tools that

some participants may not be aware of. Done via a

combined lecture and web based learning system.

Subject specific teaching – rapidly introducing the

participants to the basics of the field in which they

would be performing the research. Done via a

combined lecture and implementation format.

Graphics programming unit programming – ensuring

that everyone could within a very short time produce

effective parallel implementation of simple image

processing algorithms. Done via a formal lectures

and workshop format.

The third ideal was that topics for the various research

projects had to be selected very carefully to ensure that the participants at least had a reasonable chance of achieving the

2After the 2009/2010 IP Challenge a student was heard to

comment that winning was akin to achieving an extension of a

sentence. The value of a research assistantship in the image

processing laboratory for one year is ZAR 40,000 (approx.

USD 5,000) for roughly 300 hours of work.

set goals. For this reason the participants were only allowed to select between given topics based on recently published journal papers.

All of the above had to be included into a specific structure that would guide the full 6 week course and is presented in Table III. (Table II indicates the content while Table III indicates the full curricular structure.)

The third element was that participants in the IP Challenge would be paid a very small stipend – the equivalent in South Africa of being paid the minimum legal remuneration for a full time job. While we wanted to ensure that the participants were earning a small income, it was not meant as the major hook to get applicants into the program. It must also be mentioned that the total budget for the 2009/2010 IP Challenge was less than ZAR 80,000 (approx. USD 10,000).

It is also important to note that the expectations of the authors were that the pressure on the participants would preclude any additional time being invested outside the class and/or laboratory space during the first six weeks. Furthermore it was expected that during the second period, the 7th to the 12thweek, the participants would spend only 10 hours per week in the laboratory – because the first teaching semester of the new academic year would have commenced. The second period, after the formal close of the IP Challenge, was arranged so that the participants would still have access to the laboratory (and mentors) for completion in their own time of work towards the publication – even if they were not selected as winners.

TABLE III. IP CHALLENGE CURRICULUM

Week Program structure

Main aims Presentation

format

Time

(hrs)

1 Basic research background

and processes

Lectures and

practicals 50

2 Selecting and developing a project description

Lectures,

practicals and

group discussions

50

3

Background reading, searches for additional

material and background

technique studies

Mentor meetings,

research feedback,

help with material to be become

expert in

50

4 Development of theoretical basis for new project

Mentor meetings 50

5

Implementing solution –

firstly via prototype and then

via implementation software

Mentor meetings 50

6

Solution testing and

statistical analysis

First article drafting

Mentor meetings

Guided writing workshop

50

7th -12th week

Finalization of article with mentor

Structured

feedback and interviews with

mentor

100

TOTAL 400

D. The idealized agents

Our own experience over more than 40 years of engineering teaching and research is that there seems to be

little correlation between undergraduate GPAs and research ability. Some of our own best performing postgraduates come from at best mediocre undergraduate backgrounds. Furthermore we were convinced that most of the students we wanted on the course would not previously have considered themselves as potential researchers.

Confrontationally, at least at our university, we threw open the access to students from the Faculty of Science as well as the Faculty of Engineering on the principle that students would learn well from others with different educational viewpoints. We thus advertised the IP Challenge within UJ via word of mouth and simple flyers in 2009/2010.

More applicants arrived than we could cater for and it was thus necessary to develop a selection process. This engendered a great deal of discussion, and no real consensus. Finally it was decided to give the first IP Challenge the least likely chance of success and go with a very controversial selection model. We deliberately selected students who we felt would never have been considered by our colleagues as potential postgraduates for exactly this reason. We were attempting to create a scenario in which both the mentors and participants would initially be expected to fail by their peers3.

The selection process has therefore developed along the following lines:

Select via individual interview between the student

and at least two of the senior mentors – and only if

there is some doubt would the academic history be

considered.

Select to prefer second or third year academic

students rather than fourth year students

Select to ensure diversity (racial, gender and academic

background) – and then enforce this diversity within

the small groups.

Select to create potential conflict – not all conflict is

bad and often students will learn more about

themselves in a difficult situation.

Select to make the possibility of success small –

conquering a research project should be no less

exhilarating than other extreme sports.

Notwithstanding the above, the first IP Challenge was run with 6 selected students (from more than 20 applicants) and the second had 10 students (from more than 20 applicants)4.

As we were certain that the success of the IP Challenge would revolve around the interaction between participants and the senior researchers who would be acting as mentors, it was decided that the mentors also had to commit to a set of expectations. While the time required for the involved staff of 40 hours per week may seem to be excessive it was considered a minimum requirement to take part. A complete set of expectations for the staff involved is:

3For some obscure reason failure is a dirty word in tertiary

education. 4At time of writing we again have more than 20 applicants for

the 2011/12 IP Challenge and again intend to select only eight.

Be accessible for the full duration of the initial six

weeks. (In South Africa the IP Challenge bridges the

summer holiday period – including the Christmas to

New Year break.)

Be available for at least an hour per week during the

second period for consulting with the group and

helping with paper reviews.

Be prepared to mentor inexperienced researchers who

might need more directing than would be expected

from postgraduate students.

Be prepared to join a certain number of the classes to

ensure that the participants could occasionally hear

answers by the mentors to questions asked about

research process or methodology or image processing

fundamentals.

E. Idealized outputs

Finally, we listed what we considered the minimal set of outcomes that would serve to introduce young engineering students to research as experienced by senior research students or active research staff. From the first it was expected that students taking part in the IP Challenge should be able to achieve the following outcomes:

1. Development of research skills not normally

present in undergraduates.

2. Development of study and analysis techniques not

normally present in undergraduates.

3. Improved performance by the participants in the

final year undergraduate research course that is

part of the standard engineering curriculum.

4. The publication of a research quality paper for

submission to a potential international peer

reviewed conference by the groups of two. 5. Increased student academic self-confidence and

self-esteem.

The idealized outcomes for the participants and the senior researchers and mentors were obviously not the same. As the IP Challenge mentors discussed their expectations it was clear that they were slightly more ambitious about the outcomes. These can be summarized as:

1. Making the exercise so stimulating that significant

numbers of the participating students would

consider further research degree registration after

completion of their undergraduate degrees.

2. Instilling a new excitement amongst all of the

students and staff within the research group when

confronted by the performance of the juniors,

resulting in a general increase in research output

from all group members.

3. Testing our expectation that by making the contact

between undergraduate, postgraduate and active

researcher staff intimate and continuous, the

younger researchers would gain the most possible

experience in a short period.

IV. IP CHALLENGES 2009/2010 AND 2010/2011

After the extensive analysis of our idealized undergraduate research course we realized that our vision would not allow it to function optimally within the lectured time of the normal undergraduate program. Courses are too full, there are too many competing events, examinations and other evaluations are an ever present distraction to the students.

So we realized that to develop what we saw as the ideal course, we would need to place it outside the normal curriculum time and away from the distractions that render most present undergraduate research courses less effective than they should be.

We thus commenced with the initial IP Challenge in November 2009 and with the second course in November 2010. These two courses will now be discussed in terms of the idealized expectations that we had aimed for.

TABLE IV. DEMOGRAPHICS OF THE 2009/2010 AND 2010/2011 GROUPS

Measure Student demographics

Gender Male

Female

10

6

Ethnicity White – 8 Black – 6

Asian / Indian – 2

8 6

2

Faculty Engineering

Sciences 8 8

GPA levels

x < 3

3 < x < 3.25

3.25 < x < 3.5 3.5 < x < 4

5 (lowest = 2.5)

5

4 2

Academic

level

Second year

Third year Fourth year

10

4 2

A. Participant demographics

As per our defined aims for agents the participants were very diverse. Table IV indicates exactly how diverse the groups were after the selection process was completed. Furthermore the groups were not allowed to be self-selected but were setup to maximize the intra-group diversity.

Groups of two participants were allocated to a research group for the duration of the IP Challenge. As far as possible, groups were deliberately selected to include different genders, different ethnicity and different academic background and performance levels. In both years the groups managed to perform adequately even with this improbable composition model. While there was some uncertainty about this arrangement amongst group members at the outset in both years there were no instances of groups disintegrating and having to be reformed with other members.

B. Actual Implementation

Contents for the various courses listed in Table II were developed on an ad hoc basis as required for the initial IP Challenge. After the first IP Challenge it was realized that a more formal approach to the notes and the courses would be needed and this happened for the second round. Especially

material around research methodology and research process were required that were pitched at the level of our undergraduates rather than the normal texts for postgraduates.

Mentors had in the 2009/2010 underestimated the amount of time involved in the program. We found that the senior staff and PhD students were spending at least 6 hours / day for the first two weeks and then about 12 hours per week. The junior mentors (predominantly masters degree students) spent as much time in the beginning but continued spending in excess of 25 hours per week throughout the final 4 weeks.

All the mentors reported spending more than 2 hours per week for the remaining 6 weeks of the second round while some report more than twice this.

From the 2009/2010 IP Challenge it was clear that the participants were struggling with the presentation rate of the material and that to compensate they were arriving early and staying as late as we would allow the laboratory to remain open (0700 – 2200 was the laboratory open hours).

Groups had struggled in 2009/2010 not because of the workload but because of their own self-inflicted expectations such as working excessive hours voluntarily. They had been warned in a formal briefing at the commencement of the IP Challenge that what we as the senior staff expected was unheard of at academic institutions and that we fully expected them to fail at the goal since many full time postgraduate students do not meet this target within one year.

C. Actual Outputs

Outputs that we have achieved from the first two challenges can be summarized as follows:

15 out of 16 participants completed the full 12 weeks (both periods) – the one who did not complete has requested a second chance

4 participants have been awarded research assistantships for a year in the image processing research group – and all have completed this assistantship period as useful 5 members of the group

2 participants have already moved on to full time postgraduate research programs (and have volunteered to be mentors in the 2011/2012 IP Challenge)

3 blind peer reviewed international conference proceedings have been achieved by the participants from the projects they were assigned

2 additional peer reviewed international conference proceedings have been achieved by the participants from the past IP Challenge members during their research assistantship years – this time with minimal help from mentors or senior staff

5 “Useful” here meant adding to the intellectual output of the

image processing research group

1 participant has already published (and co-published) 3 conference proceedings and/or articles since joining the program – all before achieving a master degree in IT

Postgraduates in the image processing research group are seeing juniors publish and are being encouraged to do the same

The senior postgraduate students who have acted as mentors have experienced the joys (and tribulations) of being research supervisors much earlier than would normally be true – and have developed a more clear sense of the place of the various research components

The image processing research group has shown growth as a result of the research staff being labeled “approachable” by the participants as well as bystanders

Other groups in the Faculty of Engineering are starting to experiment with similar programs

D. Some mistakes we made

Not everything worked as well as we would have liked. Especially in the 2009/2010 Challenge we underestimated the ways that university administrations would handle something as unusual as the IP Challenge. Simple issues such as issuing student cards for library and building access ballooned out of all proportion6.

When asked why we were paying “students” for what seemed to be a class we were forced into a convoluted description of what was happening – undergraduates doing research? “They are not required to do research so why pay them?” Getting earlier buy-in from the Faculty and UJ administration would have been preferable.

We underestimated the enthusiasm of the more determined participants. We were buttonholed on why they could not have longer access to the laboratory – both during the normal week, weekends and the mandatory Christmas-New Year break. We attempted to create access to the laboratory far beyond UJ regulations only to be stymied by the protection services and the administration.

We missed a golden opportunity to increase postgraduate and undergraduate contact in the first IP Challenge by placing the participants in a separate laboratory – arguing that the noise and undergraduates would bother the research group as a whole. The second time we realized that integrating them from the outset would make it even more real for the participants and also allow the postgraduates who were not formal mentors to get a feel for the project.

6 An unbelieving administrator argued that “students would

never want to be on campus between Christmas and New Year

when everyone is taking a holiday.” So why issue special

access cards?

V. RESULTS

We have completed two full IP Challenge iterations and are able to report on some of the results which we are confident will hold into the future. These findings will be reported against the same set of idealized parameters as introduced in Section II.

A. Inputs and Contents

The development of the formal teaching contents required substantially more effort than had been expected by the senior mentors. While postgraduate research students can to some extent be “pointed and let loose” the undergraduates needed help in finding first sources. The participants were eager but had need of more support and direction than what we were used to7. As a result the senior staff report working more than 40 hours per week during both the first and second IP Challenges ensuring that materials and problems for practicals were properly developed and presented.

As a further result we have developed a substantial set of aids for teaching research to undergraduates that have now been tested on two groups. These aids include selected journal papers that are in specific ways either exceptional or deficient – an essential requirement for teaching the critical reading and research hypothesis sections.

Developing the testable questions for the research process training also took considerable effort. The questions had to have non-obvious answers that were resistant to simple analysis and could only be proved by the participants by developing numerical experiments. Example questions revolved around whether it is possible to decompose large spatial dimension Gaussian filters for image processing applications with smaller ones.

As a further result we have developed a substantial set of aids for teaching research to undergraduates that have now been tested on two groups.

B. Process and Agents

Attracting minorities to research, especially in the engineering field, is at best a difficult issue [19] that is often seen to be made more difficult by the predominance of researchers who are white, male and aged. Furthermore there is a perception that as professors attract students with high GPA values, students from academically disadvantaged background will be underrepresented in a recruitment drive for undergraduate research students [19]. We have managed, we believe, by using a spread of mentors from different backgrounds and ages to have allowed a certain level of identification between the participants and the younger researchers in the group as a whole. Of course having the IP Challenge participants in the one corner of the same laboratory space as the regular postgraduates has also ensured that there

7The two most senior staff involved have supervised more

than 60 postgraduate students during the past 20 years – and

had used this experience as a yardstick for what we expected.

are role models all over – some looking smug (they were there last year!) but all willing to smile and help if someone is stuck.

The annual winners were selected by the supervisors and mentors from all the participants by a rigorous process of reviewing both the paper outputs as well as general research abilities developed. Interestingly in both courses so far the two winners have come from different groups.

The deliberate stress and the enforced diversity related to the selection methods and the group allocation methods were handled better by the participants than any of the mentors could have predicted. We had attempted to ensure failure by in fact stacking the odds against the participants but in fact this seemed to spur them on to work harder.

Given the curriculum and the upfront nature of the challenge to deliver a research quality paper in a short time it is not surprising that the participants felt overwhelmed at the beginning and the end of the six week first period. For a short while in the middle three weeks some of the groups became overconfident and were surprised at how difficult (unpredictable and non-mechanistic) it is to manage a research project, this compared to formal studies where the management is applied by an external agency. The level of control and the amount of externalities that needed to be managed conspired to present all of the groups with what seemed insurmountable difficulties at various times.

One unexpected consequence was a reduction in the academic performance of the participants (now students again) in their courses during the succeeding semester. The last six weeks (of which three overlapped with formal teaching in the first semester) was experienced as no less intense than the first six weeks. The pressure to finalize the paper eventually began to tell and this interfered with formal course work. After the first challenge we were aware of this and actively worked via the mentors to ensure that the students realized what the correct priorities were8.

An expected side effect for the senior staff was that our normal research work almost stopped dead during the first two weeks when the course material was presented. The intense level of interaction, lecturing and assisting in the practicals made for an exhilarating but exhausting time. Even during the second presentation of the IP Challenge we found that our other work was relegated to late at night before the next day’s course.

From a post course set of interviews with staff, mentors and participants we can also report on the benefits that the IP Challenges view as important. Table V lists these benefits in order of importance.

It is interesting to note that while there is some correlation between entries in Table V and Table I it is by no means perfect. Items that participants in the IP Challenge felt were very important (inspiration and understanding the research

8Although as researchers we were delighted with this we were

concerned that some students would actually fail their final

year and not be able to register as postgraduates during the

coming year.

process) do not figure prominently in Table I, while benefits staff perceived as having the most value are unique to Table V (e.g. coping with work-loads and rates at which material must be mastered) – and can both be considered important attributes of modern researchers.

TABLE V. PERCEIVED BENEFITS OF AN UNDERGRADUATE RESEARCH

EXPERIENCE

Benefit Reported benefits of the IP Challenge

Mentor perspective Participant perspective

1 Learn to cope with

high work rates Felt inspired

2 Learning to work

independently

Understanding the research process

3 Maturity of the

participants

Learning to work independently

4 Improve oral

presentation skills

Develop a continuing relationship with faculty

5 Improve written

presentation skills

Learnt to focus intensely on a single issue

6

Learn to find and use

scientific literature

with minimal help

Enhancement of study methods

7 Learnt to manage a complex project

C. Participant feedback

In qualitative research based interviews with a third party (under a guarantee of anonymity) some participants’ responses were elicited so that future IP Challenges could be more effective as research training ground. Responses from them include (actual quotes and paraphrases from interviews):

In the beginning I wanted to leave because I thought “surely it is impossible to meet these targets – it was overwhelming”

“Inspirational but demanding” – “…seemed like there was no chance of me doing this…”

“Left formal taught courses feeling boring”

“Grueling, never been challenged to the same degree…” – “classes are dreary and progress slow (sic) compared to the material in the course”

“…and then I’d try it [own idea] and I’d fail. I’d think to myself: Now why is it not working? … Finally I realized that the process, follow the steps, read, check … was the only way to get results”

“Real downer going back to classes in February”

D. Mentor feedback

In further interviews with a qualitative researcher mentors were given the opportunity to give their views on the first two iterations also so that future IP Challenges could be more effective as research training ground. Responses from them include:

Uncertain whether the mismatching of participants is useful to achieve research outputs

“Winners have been delivered – and the hosting research group has grown in size and confidence and competencies not normally associated with postgraduates”

“Allowing the undergraduates to stay during the semester has led to some crowding”

“Students who did the optional extra practicals really excited us all”

“Groups compensated internally for mismatches in abilities”

Surprise that high GPA students are not the winners – and that sometimes they are even the students that struggle with the independent work elements

“Some students need their hands held”

“Some students were so focused they did not understand what the project was really about”

“Where students managed to establish a relationship with a particular mentor, it seemed to benefit their progress on the project and increase their confidence” (strongly supports one of the primary results reported in [1])

VI. LESSONS LEARNT

In summary there are some lessons for persons involved in undergraduate research that we would like to highlight:

Students are willing to work harder than expected IF mentors are involved. The authors were surprised by the level of commitment amongst participants that during normal study periods would balk at 45-50 hours per week but were clearly spending in excess of 60 hours per week without any complaint.

Undergraduate students can produce useful research outputs once schooled in research as process without substantial direct interaction on the project.

Courses such as the IP Challenge can motivate both staff and senior students when confronted with the enthusiasm and success of the undergraduates.

Winning is more important than we thought it would be for the participants – it is not certain which is more important to the participants – the research assistant salary, the “bragging rights” or the permanently allocated workstation in the image processing laboratory.

Having 3 opportunities for oral presentations within a safe but critical environment leads to

dramatic improvements in the ability to present research results.

Participants when deliberately mismatched can still function effectively and learn to manage the group’s abilities to ensure success, often by the stronger member taking responsibility for the more complex and challenging aspects of the project/research.

VII. THE FUTURE OF THE IP CHALLENGE

We are getting ready for the third presentation of the IP Challenge from November 2011 to January 2012.As before we will be adjusting the course slightly, making use of feedback from mentors and participants from the previous years.

We will this coming year add some new quirks to the challenge to attempt to better understand some of the parameters. For instance, we are this time doing formal pre and post course interviews with all participants in conjunction with a researcher from the Faculty of Education. The aim is to try and gauge the undergraduates’ perception of what the research project will contribute to their personal growth as opposed to their career growth (the basis for most studies into undergraduate research).

There will also be some formal personality type testing to determine whether it is possible to select the groups with even more challenges to work through in order to increase the odds of group failure rates even further.

There will also be a concerted effort to make the course even less accessible in terms of effort and an effort to make winning the IP Challenge even more prestigious. A “bragging board” has been mooted as a way of creating a sense of permanency for the IP Challenge contenders.

To ensure that the groups meet the primary goal of publishing we will also increase the mentoring for writing and presentation further in the coming event. We will aim for an intermediate presentation shoot-out that should act as a focusing event somewhere in the fourth or fifth week to prevent the middle of program doldrums we noted above.

It has been decided to implement a formal debriefing session at the end of the first six weeks. This would serve two purposes – firstly to make clear the level of progress achieved by all (not just announcement of the winners of the research assistantships) and secondly make it possible to provide final career and course guidance. At this point some of the mentors and staff have a better insight into the participants’ strengths and weaknesses than they themselves have.

There are however some items we do not wish to change, primarily around the small groups, the close – even intimate – interaction with staff and senior postgraduate students, the mismatching of group members and the reliance on a strong teaching component.

Perhaps the last word should come from a student interview response in [19] “As one student reported when asked to give advice to an incoming computer science student, ‘My advice is to find a faculty member, who will tell them what to do, who is

interested in them, and will get them motivated to pursue research. I am a person who wants to do that.’.”

ACKNOWLEDGMENTS

The authors would like to thank Prof E de Kadt (ED at UJ) for extensive discussions around undergraduate research projects as well as Prof H Kriek (DVC Finance at UJ) for financial support of the project. We would also like to thank the anonymous reviewers for interesting feedback that will certainly change future IP Challenges.

REFERENCES

[1] A.L. Zydney, J.S. Bennett, A. Shahid, and K.W. Bauer, “Impact of undergraduate research experience in Engineering”, Journal of Engineering Education, pp. 151–157, April 2002.

[2] J.W. Prados, G.D. Peterson, and L.R. Lattuca, “Quality Assurance of Engineering Education through Accreditation: The Impact of Engineering Criteria 2000 and Its Global Influence”, Journal of Engineering Education, pg 165-184, January 2005.

[3] D. Lopatto, “The essential features of undergraduate research”, Council Undergraduate Research Quarterly, vol. 92, pp. 139–142, March 2003.

[4] C. M. Kardash, “Evaluation of undergraduate research experience: Perceptions of undergraduate interns and their faculty mentors”, Journal of Educational Psychology, vol. 92, pp. 191–201, 2000.

[5] M. Healey, A. Jenkins, “Developing undergraduate researchand inquiry”, The Higher Education Academy, June 2009

[6] C.A. Merkel, “Undergraduate Research at Six Research Universities: A Pilot Study for the Association of American Universities”, Califonia Institute of Technology, 2001.

[7] A-B. Hunter, S.L. Laursen, and E. Seymour, “Becoming a Scientist: The Roleof Undergraduate Research inStudents’ Cognitive, Personal,and Professional Development”, Science Education, Wiley, 2006.

[8] P.L. Chinn, and M.K. Kramer, Theory and nursing: A systematic approach. Mosby Year Book, 3ed, 1991.

[9] A. Botes, ‘n Model vir wetenskapsbeoefening in die verpleegkunde [A model for science practice in nursing]. Unpublished doctoral dissertation. Johannesburg: Rand Afrikaans University, 1989.

[10] T.A. Schwandt, A Diagnostic reading of scientifically based research for education. 2007.

[11] L. Krefting, “Rigor in qualitative research: the assessment of trustworthiness.” American Hournal of Occupational Therapy, vol 45, March 1991, pgs 214-222, 1991

[12] J. W. Creswell, Research design: qualitative, quantitative, and mixed methods approaches. 2ed University of Nebraska, 2007.

[13] E. Jupp, “Participation, local knowledge and empowerment: researching public spaces with young people.” Environment and Planning A, vol 39, pgs 2832-2844, 2007.

[14] R.E. Stake, “Case studies.” In Norman K. Denzin & Yvonna S. Lincoln (Eds.), Handbook of qualitative research (pp.435-453). Thousand Oaks: Sage, 2000.

[15] W.A. Clarke, N Clarke and A.L. Nel, “Structured guidelines for novice researchers to produce their first peer reviewed academic paper”, IEEE Educon 2012, in press, 2012.

[16] L. Reisberg, “Research by undergraduates proliferates, but is some of it just glorified homework?” Chronicles of Higher Education, 1998.

[17] S.H. Russell, M.P. Hancock and J. McCullough, “Benefits of undergraduate research experiences”, Science, vol 316, pp. 548–549, 27 April 2007.

[18] K.W. Bauer and J.S. Bennett, Alumni perceptions used to assess undergraduate research experience, Research Report, University of Delaware, Undergraduate Research Program, 2000.

[19] A.Q. Gates, P.J. Teller, A. Bernat, N. Delgado, and C.K. Della-Piana, “Meeting the challenge of expanding participation in the undergraduate research experience”, 28th AnnualFrontiers in Education Conference, 1998.