Approach for Evaluation of a College and Diploma Program for Advanced Standing Entry to a Degree...

Exploring Innovation in Education and Research ©iCEER-2005

Tainan, Taiwan, 1-5 March 2005

Approach for Evaluation of a College and Diploma Program for Advanced Standing

Entry to a Degree Program

Timothy L.J. Ferris

[email protected]

Systems Engineering and Evaluation Centre, University of South Australia, Mawson Lakes, South Australia,

Australia

ABSTRACT: Recently there has been a shift in the educational background of overseas applicants for entry to Bachelor of

Engineering awards in Australian Universities. In addition, the 1988 Dawkins Higher Education policy proposed that Australian

higher education in general should be structured to enable individual people to progressively upgrade qualifications throughout their

career, with minimal loss of time compared with choosing a minimum time pathway to their ultimate educational goal from the

outset. These background factors have resulted in Australian universities being confronted with a need to assess holders of various

Diplomas in engineering disciplines with a view to granting entry to a Bachelor of Engineering award accredited by Institution of

Engineers Australia and consequently throughout the Washington Accord countries as a first award in engineering, with substantial

advanced standing based on the preceding studies. The numbers of applicants and the contextual factors result in a need for

Australian universities to establish advanced standing agreements with diverse ‘feeder colleges’ to the effect that a holder of a

particular diploma with a certain standard of attainment will be granted a certain advanced standing. Particularly where the ‘feeder

college’ is overseas there are complications to the process related to distance in both geographic and cultural dimensions. The author

has been involved in a task requiring determination of a generalisable set of criteria by which to assess the quality of education

achieved in a program of study involving a process to measure student attainment against a defensible standard, in this case Bloom’s

taxonomy of educational objectives, college academic staffing and facilities, in addition to the details of course and program content.

The discussion of the paper will provide basis for people concerned with: 1. Accreditation of programs for recognition as

engineering awards; 2. Recognition of advanced standing entry to accredited programs; and 3. Educational development assistance.

INTRODUCTION

Australian universities have received a flow of overseas

students over a very considerable period. During the history of

overseas students entering Australia there has been a series of

changes affecting the kind of people seeking to come to

Australia to study and the motivations prompting their

movement. This paper addresses certain issues arising from

some of the more recent changes in the overseas student cohort

coming to Australia and opens more general discussion of

issues arising from the point of view of curriculum, understood

in a broad sense as the entire set of educational activities and

arrangements established, in their context that lead to student

learning.

The changing context of overseas countries from which

overseas students come and changes in the Australian

educational system have resulted in many students studying in

their home country to attain a sub-degree level qualification in

a field related to a Bachelor award in engineering. Such

students then seek, and are, subject to various conditions

granted advanced standing that significantly reduces the

amount of study that they need to complete in Australia in

order to fulfil the requirements of a Bachelor of Engineering

award. This phenomenon requires that Australian universities

develop means to assess the quality of the various colleges and

their diploma awards in order to assess the implied equivalence

to parts of the universities’ own bachelor awards. The

importance of this issue has grown with the combination of

increasing absolute numbers of applicants and the fact that the

predominant academic pathway sought by intending overseas

students of engineering in Australia involves this two stage

process.

The issues discussed in the paper are important for the

consideration of the matter of accreditation of engineering

awards because the issues open significant issues concerning

the nature of engineering education and the qualities that a

person must develop in order to be an effective professional

engineer.

CURRICULUM

One, simple, view of curriculum sees curriculum as concerning

the itemisation of the substantive content, or topic areas, to be

taught through a program of studies. Such a simple view of

curriculum is common in many places, with engineering

academics being an example of a large body of participants in

education who hold to the simple view. Examples of such

usage of ‘curriculum’ by engineering academics can be seen in

the following references amongst many others that could also

have been cited [1-20]. In various ways these authors reflect a

view of curriculum as concerning the topic areas to be covered

by a program of study in order that that program of study will

lead to its graduates being qualified appropriately for a certain

field of activity. The emphasis of such work is on what the

student must learn about in order to be worthy of graduating in

the nominated field. However, there is much more to being

competent to perform significant work in a field than just

having knowledge of a certain set of topics.

However, one should note that the authors cited above come

from the USA, and date between 1955 and 1981. This range of

dates is important for two reasons, it pre-dates the diffusion of

more modern, and holistic, concepts of ‘curriculum’ and

because the background of the majority student group was

relatively homogeneous. The importance of the homogeneity of

the background of students is significant in the matter of the

holistic interpretation of curriculum because where the

background of students is homogeneous it is possible for the

educator to make considerable assumptions about the pre-

existing knowledge of the student that are reasonable, and are

consistent across the class. Any assumption that is consistent

across the class permits the educator to take that particular

matter of competence as given, rather than as a matter that

must be supported by deliberate educational action.

Consequently, in situations of homogeneous cultures, in which

all, or almost all, students bring very similar background

experience and knowledge to their studies it is practical for the

educator to assume the student background and to omit

processes for the development of such skills from the goals of

the educational process conducted.

In the more recent educational literature ‘curriculum’ has been

presented as concerning the holistic presentation of learning

experiences and context to the student [21-29]. Curriculum

includes the content of the material presented to the student, as

in the earlier conception, but also includes the manner in which

it is presented to the student, the learning tasks the student is

expected to perform in the process of the teaching and learning

engagement with the material, the educational institution

context including the physical and intangible resources

available to students, the facilities available for ‘extra

curricular’ activities such as clubs and societies, and the social

milieu created by the other students. The argument to support

adoption of this broad interpretation of curriculum is that the

presence and effectiveness of these factors affects the learning

that the student achieves through their program of study.

To some extent there has long been an intuitive sense that these

‘extra’ things are significant, with the response being the

establishment and strength of schools of the form of the

English ‘public schools’, which in addition to a reputation for

good academic standards, in the narrow sense above, also

provided opportunities for students have a broad range of

experiences that lead to development of the whole person in the

context of learning about particular academic content. The

intuitive sense concerning the value of a holistic, broad,

curriculum has been demonstrated through the willingness of

generations of parents to make significant sacrifices to send

their children to such schools in the expectation and belief that

their children would be developed in a broader range of ways

through that experience than they would have been had they

been sent to a school offering only a required content based

curriculum.

In post secondary education the appreciation of the holistic

purpose of education, being to develop the person to be a

person able to exercise certain kinds of skills and to bear

certain kinds of responsibility related to the exercise of those

skills seems to be little appreciated. The two major situations in

which the holistic purpose of professional education is clearly

acknowledged and the consequences of that acknowledgement

are worked through into practical expression in the practice of

the education are in military academies and theological

education. In both these fields, as different as they sound, there

is a need to develop students so that as graduates they will be

able to make decisions significantly impacting others. The

graduates must exercise considerable personal maturity and

judgement at very early career stages. In theological education

this is reflected through the acknowledgement of the

curriculum as requiring broad ranging development of the

student to develop the student in knowledge and personal

qualities to be suitable to practice ministry [30, 31]. In the

military academy case the graduates will make life and death

decisions affecting their own military unit and with potential to

calm or inflame passions influencing international politics.

Although engineering does not have such stringent personal

requirements of its practitioners it is appropriate for

engineering education to use a broad understanding of

curriculum because successful practice in engineering is

dependent upon a combination of knowledge, practice of

personal responsibility and possession of a broad range of

skills such as effective inter-personal and communication

skills. A narrow view of curriculum sees education in each of

these matters being achieved by means of courses focused on

each of the particular matters, whereas a holistic view of

curriculum sees many of the desirable personal qualities being

developed through the means by which other parts of the

curriculum are taught. Thus a holistic view of curriculum sees

that both the content and the means and context of the teaching

are important to the educational outcomes and some desirable

outcomes are understood as being achieved through the means

of teaching and assessing other desirable outcomes.

DAWKINS REFORMS IN AUSTRALIA

In Australia during the 1980’s there was political pressure

concerning the cost and status of post secondary education. In

the Australian education the then major supplier of funds for

post secondary education was the government sector, with the

federal government responsible for the provision of the funds

expended through tagged grants to the states even where the

states had formal authority to provide for the education. The

result was that funding for post secondary education was seen

as a federal issue.

Australia was in a period of restructuring of the economy

during this decade to effect a transition from the former

somewhat insular, local market oriented economy with a cost-

plus mentality affecting productivity, efficiency and

competitiveness. The result was that issues of efficiency in the

provision of educational services came to be emphasised in a

new way. A further factor was that the Australian Labour

Party, ALP, formed the government at the federal level from

1983 to 1996, and the traditional power base of the ALP is the

working person and other socially disadvantaged groups.

The combination of these factors led the federal government to

look for means to change the post secondary education sector

to provide pathways to provide means for individuals with

certain qualifications and work experience which suited their

achievements and intentions at some earlier time in their life to

upgrade their qualifications conveniently and with little or no

disadvantage when their circumstances, such as employment,

made it advantageous for them to make such an upgrade. The

first relevant government publication was a study discussing

various issues associated with the efficient and effective

delivery of education and training for the strengthening of the

national competitive position [32]. Amongst the

recommendations was that post secondary education should be

organised as a series of qualifications that can be used in a

progressive upgrading process through establishing all post

secondary qualifications as part of a layered framework, where

for example a trade certificate or diploma can function as a first

step towards a higher qualification, such as a degree, without a

repetition of study time resulting in the person expending an

increased aggregate duration in study to achieve the intended

higher award.

Following this publication the official policy document was

released [33]. The Dawkins policy made many significant

changes to higher education in Australia. To further efficiency

in the sector the system would be unified into a single higher

education sector with all institutions treated on the same basis

as universities rather than the preceding three tier system of

universities, institutes of technology and colleges of advanced

education, each specialising in certain parts of the educational

and training task and funded in significantly different ways.

The former system resulted in institutions which were

perceived as significantly different in status. One objective was

to flatten the perceived difference of status. However, in the

new system, there rapidly came to be a perceived ranking of

universities following from the status of the antecedent

institutions, which in the case of most of the newer universities

was some combination of antecedents which had been

amalgamated. The old Technical and Further Education,

TAFE, sector which had provided various certificate, trade and

technician type awards continued under the Dawkins policy

with the major change being the introduction of a unified

national curriculum centred on competency based assessment,

and thus permitting students to complete or upgrade awards

whilst making career moves around the country.

The policy also, and partly through the flattening of the system

which often linked providers of lower qualifications with

providers of degree qualifications within one institution,

required that the concept of multiple pathways to degrees be

created. This contrasted with the previous thinking. Previously

there was a profound separation of people at the time of

completion of secondary school. Some progressed into trade

and technician level qualifications, usually based on finishing

school before completion of year 12. These people were then

‘locked-in’ to a career path that was based on that choice, and

would normally remain in the lower employment ranks. The

others were those who completed year 12 at school and

proceeded to degree level study and would pursue a career path

in the so-called ‘professional’ ranks. Where a holder of a

TAFE award sought entry to a university program the TAFE

award was generally assessed from the point of view of

equivalence to year 12, rather than as providing equivalence to

some part of the degree award.

The effect was a strong discouragement for TAFE award

holders from seeking to upgrade qualifications because of the

time cost brought through the lack of recognition of previous

awards by the university sector. The government saw this as

discriminatory, because social class and area of upbringing

were significantly linked to entry to university programs, and

thus against the social pillar of ALP policy, and inefficient in

the use of educational resources or the ‘artificial’ restriction on

the progress of people to higher productivity occupations, and

thus against the concern for national competitiveness.

The policy requirement for alternative pathways to degrees has

required that universities recognise lower awards as providing

equivalence of at least part of the Bachelor award offered by

the university, and so to provide substantial advanced standing

for studies performed outside the university as a step towards

the completion of the requirements for the degree. The policy

is clear in indicating that the amount of advanced standing

should result in students pursuing a pathway in the same field

of endeavour progressing through their pathway in the same

time as it would have taken them pursuing the traditional

‘standard’ pathway. The shift in the TAFE sector to

competency based assessment created considerable controversy

in the higher education sector, which would received TAFE

graduates seeking to upgrade their qualifications, because of

the profoundly different basis of assessment in the two sectors

and the related issues of the kind of capability developed in

each place.

PROBLEMS OF ADVANCED STANDING ENTRY

Advanced standing entry into an engineering degree based on a

completed technician diploma award presents a number of

problems. The author has discussed these problems in [34]

finding that the challenges presented are several and complex.

The first problem is that if the diploma award is well designed

as a qualification to make the graduates effective in a particular

kind of employment, that is, is designed to educate technicians

appropriately to their role as technicians, then it is not simply

the first half of an engineering degree. The role of technicians

in industry is to perform a range of reasonably routine or work

as directed activities that require a considerable level of

technical skill. Thus technicians are often used to perform

routine calculations and modelling of proposed products, and

to perform the hands-on work of dealing with the stuff of the

discipline in the workplace. In contrast, the purpose of the

engineer in the workplace is to have an elevated level of

analytical knowledge of the things related to the technical

discipline, and consequently the ability to make judgements

about the most appropriate solutions to particular technical

issues when selecting between a range of candidate solutions.

This role requires that the engineer develop different

capabilities through the process of professional formation.

The different capabilities required of engineers involve the

engineer needing to develop an elevated level of analytical

skill, in order to be able to do the more complex levels of

technical analysis including the relating of observations of real

states of things to the analysis of those things. The observation

of states of things may be found in either the quantitative

observation process of measurement, in which case the relation

of the observed to conclusions can often be made using

formulae, and more complexly, in the qualitative observation

of things and the recognition and drawing conclusions of

significant factors that can be found in the observations. This is

a difficult process to perform and relies on the ability of the

engineer to draw insight from observations so that appropriate

action results. As a result the education of engineers

emphasises the development of high capability in analysis of

information, the drawing of insight and the making of

decisions.

In contrast, since many employers use technicians in roles

involving close hands-on contact with equipment the training

programs need to emphasize skills of handling and working

with equipment, in the construction, testing and maintenance of

that equipment. Emphasis on processes such as troubleshooting

and repair and the trade related skills of working with the

hardware is proper to the technician kind of work, but is

omitted in most engineer education programs because the

contribution of the engineer in industry is different than that of

the technician.

The second problem is that if the diploma is designed as the

beginning of a degree the emphasis in the teaching of the

diploma will be on the subject matter that is usually taught at

the beginning of an engineering degree so that the diploma will

provide the student with a relatively smooth transition in to

their later degree level studies. In this case the emphasis in the

design of the diploma is on the desired final destination of the

more able students in the class, rather than on providing a

coherent terminal award that serves the students who do not

continue and their employers well through provision of useful

and identifiable skills at the end of the diploma.

However, there is another force at work, most clearly seen in

the case of colleges in countries outside Australia which feed a

large number of overseas students to Australia. In some of

these countries the demand for post-secondary education vastly

exceeds the number of places available in universities for

degree level study and consequently there has been a spawning

of diploma level colleges. In certain countries many of the

colleges are operated by private sector businesses, in contrast

to the government operated universities, with the result that the

college proprietors’ objectives may be skewed by the business

imperative to generate profit. In the marketing performed by

many of these colleges one major emphasis, both to assert the

quality of their college and also to pursue the interest in

potential students to obtain a degree by some means, is the

foreign universities that provide substantial advanced standing,

through formal agreements, to diploma graduates of the

college. In the extreme this has caused difficulties for the

universities so named because at times the college marketing

has provided the market with the impression that the advanced

standing is either more, or with less conditions attached, than

the university has agreed. In some cases the university has

received negative publicity resulting from the disputes arising.

Another difficulty that this educational marketplace has

generated is that it has established the diploma plus advanced

standing entry to a degree as a ‘normal’ pathway for a school-

leaver to become an engineer. The perceived ‘normality’ of this

two stage pathway to a degree appears to have resulted in

distortion of the diploma awards of the colleges to satisfy the

‘upgrade’ market at the expense of not providing a diploma

award that is well adjusted to serve the employment interests

and employers of those who complete their studies at the

diploma level. However, the recognised lower desirability of

this process compared with direct entry to an engineering

degree result in the colleges accepting a class cohort who have

achieved at a standard lower than required for university entry.

The lower entry standard demands evaluation of the fine details

of the college level education or training from a holistic

curriculum point of view rather than the relatively easy course

content based approach. The need for holistic review of

curriculum has been found as a result of the difficulty that

some students who have transferred for advanced standing

entry have experienced. Therefore as a matter of ethical

concern for the negative effects of providing too much

advanced standing it is necessary to evaluate the underlying

basis on which such decisions are made. Bloom’s taxonomy

[35, 36] of educational objectives.

BLOOM’S TAXONOMY OF EDUCATIONAL OBJECTIVES

Bloom’s taxonomy of educational objectives [35, 36] was

introduced in 1954 and rapidly found broad acceptance as a

foundation for enabling program and assessment development

thinking by educators [37]. The taxonomy was rapidly

accepted in many places because it provided an effective

means to discuss the diversity of kinds of education objective

that could be obtained in various circumstances and it provided

a sensible basis for planning of teaching and assessment

activities to match the purpose of the particular educational

activity.

The taxonomy divides educational objectives into three

domains, the cognitive, the affective and the psycho-motor

skills. The work of the original group led by Bloom addressed

the former two domains, omitting the latter as a field not

touched. In each domain the possible kinds of capability that

could be achieved are described as a hierarchy of five or six,

depending on the domain, levels of possible attainment. Most

of the levels are further subdivided into several classes of

relevant factors, which are regarded as being different but at a

similar level of complexity and hence equivalent in the

hierarchy.

The Cognitive Domain is divided:

1. Knowledge

o Knowledge of Specifics

o Knowledge of the Ways of Dealing with Specifics

o Knowledge of the Universals and Abstractions in a Field

2. Comprehension

o Translation

o Interpretation

o Extrapolation

3. Application

4. Analysis

o Analysis of Elements

o Analysis of Relationships

o Analysis of Organizational Principles

5. Synthesis

o Production of a Unique Communication

o Production of a Plan, or Proposed Set of Operations

o Derivation of a Set of Abstract Relations

6. Evaluation

o Judgements in Terms of Internal Evidence

o Judgements in terms of External Criteria

The Affective Domain is divided:

1. Receiving

o Awareness

o Willingness to Receive

o Controlled or Selected Attention

2. Responding

o Acquiescence in Responding

o Willingness to Respond

o Satisfaction in Response

3. Valuing

o Acceptance of a Value

o Preference for a Value

o Commitment

4. Organization

o Conceptualization of a Value

o Organization of a Value System

5. Characterization by a Value Complex

o Generalized Set

o Characterization

The two domains presented in this form show the idea of a

hierarchy of complexity of outcomes in each domain. Further,

it is clear that educational activity could be structured in such a

way as to result in capabilities described by any of these

possible outcomes and therefore an assessment of an

educational structure could be made in terms of the typical

achievement of typical graduates of the program with respect

to the hierarchy. In addition, the relation of elements in the

hierarchy to the capabilities required for particular kinds of

work can be seen, with the result that where an educational

program exists to support some kind of industrial need for

training and education the taxonomy presents a reasonable

analysis against which to compare the program outcomes.

BLOOM’S TAXONOMY AS A MEASURE OF CURRICULUM

Where a decision must be made to grant, or not grant,

advanced standing to an applicant for a Bachelor of

Engineering on the basis of a diploma in a related field there

must be some sound basis for the decision. The dilemma is that

the student who would duplicate their previous study in the

later award will feel at least some measure of frustration over

any duplication, especially when the tuition fees and living

expenses associated with that additional study time required are

substantial, as they are for overseas students studying in

Australia. The educational problem is that the duplication may

be perceived by the student as the student sees similar content

covered but not be accepted by the faculty on the grounds that

the non-content learning that the student is supposed to develop

through the teaching and learning experience of the course is

different. On the other side of the dilemma is the problem that

the student granted advanced standing will enter a higher level

course to share with classmates who have learned similar

content, the usual basis for advanced standing judgements, but

by a different means that results in instructor assumptions

about overall student competence moulded by the majority

cohort, those going straight through, providing a mismatch

with the competence of the advanced standing students. The

result can be extremely uncomfortable for the advanced

standing student, particularly as this student faces the

adjustment problem in a class context where the instructor does

not normally need to deal with student adjustment to a new

educational situation and is not as aware of the possible

problems for some students as is necessary to protect those

students from the risk of lack of success.

For this reason, the advanced standing question needs to

consider the holistic teaching and learning context of the

college in which the antecedent study was completed. The

holistic context includes the course content, the assessment

methods applied to the students, the college facilities, and the

staff profile. Investigation of these issues is more important in

the case of overseas colleges than for local TAFE colleges, in

the Australian context, because of the wide diversity of foreign

institutions, most of which do not have a track record known

immediately to the assessor and the diversity of educational

traditions and approaches used by the colleges, as compared

with the small number of colleges within Australia that feed

such applicants into degree level awards, all of which are

reasonably close to the experience of the assessor and all of

which are members of a known educations system.

The principle criterion for assessing the effectiveness of

diploma colleges in preparing candidates for advanced standing

entry to a degree award is to look at the kind of capability

created by the study performed at the diploma college, and to

compare this with what is achieved at various points through

the degree. The structure of Bloom’s taxonomy is a reasonable

approach to take to this assessment of the holistic development

in the student through the study of certain content.

PROGRAM CONTENT

To practice design engineering any person needs to have an

effective knowledge of a significantly broad range of technical

topics at a depth that the person is able to apply all the

knowledge possessed, know what ever else needs to be known

to complete the design and capable to learn what is needed, to

create a new design that will satisfy the need and the ability to

evaluate and criticise the design achieved in order to determine

appropriate modifications of the design. Mere knowledge of

the standard textbook presentation of the technical topics, and

the ability to perform well in examination of the student’s

ability to know and describe that content is insufficient to

determine that the student is competent to practice as an

engineer. The lack of correlation between success in

coursework courses and project courses within an engineering

degree is an aspect of the phenomenon described above [38].

Fields of engineering not directly related to the design of new

products require similar capabilities to know, use and apply

understanding of technical matters.

In the work concerning relative performance in coursework and

project courses [38] it was reported that students showed

significant differences in their average mark in all coursework

courses of the Bachelor of Engineering award compared with

the final year project courses of the award. This finding could

be criticized by questioning whether the expectation of

required performance on the part of the student to achieve a

certain mark was significantly different between the two

classes of course. There is no absolute means to eliminate the

potential problem caused by this criticism. However, in the

same work [38] it was found that students with an Australian

“permanent home address” in the University record system

tended to increase in performance, and so achieved a typical

result one grade, approximately 10%, better in the project

course compared with their own result in the coursework part

of the degree. In contrast, students with a “permanent home

address” outside Australia achieved a typical result one grade,

approximately 10%, lower in the project course compared with

their own result in the coursework part of the degree. The fact

that a bidirectional correlation of a factor to relative

coursework and project result could be identified is an

indicator that difference of gross assessment expectations

between the kinds of course was not the dominant factor.

The particular correlation of “permanent home address” and

relative performance in particular kinds of course raises the

question of possible racist bias. The principle defence against

this question is that a student’s status as an overseas student is

not obvious to academic staff on any documentation used in the

assessment process, and secondly, that by appearance and

ethnicity the diversity of the Australian population and

Australian resident and citizen student cohort is difficult to

distinguish from the overseas student cohort.

Consequently the correlation of results to “permanent home

address” must be related to the difference in the way in which

the students satisfy the expectations of academic staff for

assessment of their work with a certain grade. Clearly the

overseas students are more able to match the expectations

when performing coursework courses than when those same

students are performing the relatively open-ended project

course. This is, following Biggs [39], suggestive of two factors:

1. That overseas students have experienced a learning culture

resulting in different competence to deal with open-ended

project type problems; and

2. That the manner of education in the courses that the

overseas students had actually studied whilst students did

not motivate and enable them to approach the open-ended

project situation effectively.

Biggs argues that, contrary to a popular stereotype of Asian

origin students, such students regard western education as

valuable, and so are willing to make major sacrifices in order to

obtain such education. In addition, Biggs notes that whilst such

students often have difficulty learning to deal with the open-

endedness and uncertainty produced by the Australian teaching

styles and expectations placed upon students, once the

difficulties have been overcome, the Asian students often

become excellent performers in such ill-defined teaching and

learning situations, and not infrequently are represented among

the top performers.

The curriculum issue clearly concerns not only the content,

information, that is taught and learned but also the personal and

professional qualities developed in the student as a result of the

educational experience and environment. Therefore, whilst the

fact that the content of engineering education must address an

appropriate range of engineering discipline knowledge to

enable service within one of the fields of engineering, the

manner in which that content is taught and the kind of

capability with respect to Bloom’s taxonomy of educational

outcomes that is achieved must be considered.

In much of engineering education the emphasis is on the

attainment of cognitive qualities that map to levels 1 to 4 in the

Cognitive Domain of Bloom’s taxonomy. Thus students are

typically taught methods and content that enables the analysis

of phenomena and technology of interest to the particular

engineering discipline. Such knowledge is useful for people

working at the level of technical work as directed by a senior

engineering leader. In such a situation the major contributions

made to the product architecture and technical strategy are

made by the technical leader and the ‘engineers’ do analysis

and other technical work associated with investigation of the

hypotheticals that the technical leader has proposed. However,

‘engineers’ whose skills are limited to the Bloom cognitive

domain levels 1 to 4 are not able to confidently and effectively

propose new product ideas and techniques and perform all of

the tasks required to bring those ideas to fruition.

The special gap in the cognitive abilities of such people

pertains to levels 5 and 6 of the hierarchy. The additional

capabilities require skills in synthesis, the construction of new

sets of ideas that are novel, unprecedented in the experience of

the person, and the skill of evaluation, the ability to effectively

and efficiently criticize one’s own proposal, or that of another

person in relation to the appropriateness of the idea for a

particular application. The primary expression of synthesis in

engineering is in the construction of sets of ideas associated

with actual or possible things and the evaluation concerns the

critical testing and evaluation of those things to determine that

they are suitable solutions to the problem at hand.

In the analysis of a college it is necessary to delve into the

description of the curriculum to determine what is meant by the

headings of topics covered to determine how those topics relate

to the development of the higher order skills required for an

effective engineer.

ASSESSMENT

It is well-known that people respond to any incentive scheme

that is offered to them by modification of behaviour in order to

maximise return for effort. This issue is a potential problem

with respect to quality systems in production [40]. In education

the major incentive scheme offered to students is presented via

assessment, and is the gating of students as eligible to proceed

to the next stage of education or career. Since assessment has

the gating function for the possibility of the student progressing

to the next stage of education students do what they believe is

required to achieve success according to their criteria. To the

normal student success is advancement to the next stage of

education. It is the unusual student who regards some other

objective as “success”. Such other objectives that may be

regraded by some as success include gaining a useful

knowledge of the subject area that enables the student to solve

some question that motivated their choice to study the course,

or to obtain satisfaction from developing a sense of gaining a

strong level of mastery of the subject matter.

In assessment of the effectiveness of a particular college

program in developing its graduates to be worthy of advanced

standing entry into a different educational program one must

work on the statistics of the typical member of the student

cohort, not on the characteristics of the unusual candidate.

Working with the statistically common outcome is a tractable

problem that can be found through evaluation of the

assessment processes of the subject college. The process

requires evaluation of:

1. The assessment exercises set by the academic staff. This

includes assignment sheets, examination question, practical

task sheets and other similar items.

2. Samples of the student responses to the assessment

exercises.

3. The results awarded by the examiner for the work reviewed

in 2 above.

Each of these items is important. The assessment exercises set

are important because they reflect the instructor’s expectation

concerning what students should be able to achieve as a result

of the instruction provided. In addition, the assessment

exercises reflect the perspective of the instructor concerning

what is truly important about the development of knowledge in

the discipline. The review must deduce the kind of skills tested

through the assessment processes according to the taxonomy of

those skills provided by Bloom’s taxonomy. Reasonable

assessment is likely to have elements of all the lower levels of

the taxonomy, because in order to perform any higher order

cognitive skill there is a need to have, for example relevant

knowledge. However, the important points are to note the

highest level of the taxonomy that is tested through the

questions set and the proportion of the assessment that relates

to each level of the hierarchy. In addition, it is useful to review

the balance of knowledge of specific details of the course

content and knowledge of the general principles of the course

that are required in order to succeed.

Samples of actual student assessment work are important

because they enable an assessment of the kind of work that

students are capable of producing as a result of their

educational experiences. This demonstrates quite plainly where

the educational process demonstrates effectiveness or

otherwise in the achievement of the goals implied by the

taxonomical levels of the assessment tasks.

Comparison of the work produced by students with the marks

awarded to the students for that work is important because this

shows the kind of capability that is rewarded with particular

graded results. This enables the evaluator to make judgements

about the competence of a student achieving a particular grade

from the college. This affects whether the quantity of advanced

standing awarded should be dependant on the level of pass

achieved.

COLLEGE FACILITIES

The college facilities and physical infrastructure are relevant to

the performance of student learning. The college facilities

provide the infrastructure of the learning environment.

The physical state of teaching spaces affects issues such as the

ability of students to see and hear the instructor during classes.

Therefore the teaching spaces should be sufficiently quiet and

well illuminated to provide for convenient and safe study. The

space available to students reflects the expectations concerning

factors such as the expectation of the number of reference

works that a student is expected to need to have open and

accessible whilst performing a study task.

Engineering study involves laboratory work. The amount,

quality and suitability of laboratory equipment reflects the

capability for the college to provide appropriate practical

learning experiences for students. However, in greater detail of

investigation one should determine whether the laboratory

equipment is designed only to provide examples of phenomena

operating normally, or with set piece faults, or whether the

equipment permits open development of practicals that enable

exploration of diverse effects. Generally, the greater the

flexibility of the laboratory equipment the more different

practical exercises that students can experience, and the greater

the opportunity provided for students to learn about the wide

range of things that may happen under diverse circumstances

and the subtlety of the learning that students may develop. In

electronics, for example, some laboratory equipment makers

produce systems that can be used for demonstration of a very

limited range of phenomena, and are generally quite expensive.

In contrast the laboratory could be established to use class time

assembled circuits built from basic components, and the capital

cost reduction may provide consumables for the component

based practical work for a considerable duration.

More broadly the college library reflects the learning

environment provided. Issues concern whether the library has

an adequate stock of books in the field around the areas of

interest of the students. College libraries reflect the resources

available to instructors for the preparation of their teaching

materials and to students in their addressing of assessment

questions and general learning opportunities. Where the library

is small and narrowly focused on providing access to textbooks

and other very similar material it is fair to understand the likely

educational outcomes falling short of the highest levels of the

cognitive domain hierarchy of outcomes. In addition, a broad

library reflects both the possibility for, and usually a college

environment in which there is encouragement for students to

explore issues around the subject matter of their program,

rather than to narrowly focus on the content of the courses of

instruction.

The physical environment of the college and its immediate

surrounds affects the desirability for students of staying in or

around the college throughout the day. If there are plenty of

spaces available and conducive to private study and small

group discussion the college provides an environment in which

the students may learn to work collaboratively and to use their

time efficiently. Such an environment assists students with

being able to complete study tasks using peer discussion as a

tool to assist learning, and also to learn to use inter-class breaks

efficiently for the conduct of study tasks. These skills will be

beneficial both after transfer to a university with advanced

standing entry, because these skills are typical of what is

expected and done in universities and in turn, these skills are

useful tools for career progress because collaborative and time

pressured work is typical of the workplace.

The college may provide the pleasant environment by either

establishing all the necessary facilities on its own real estate, or

by choosing a site where at least some of the facilities are

available in the surrounding community.

The establishment of a college environment that is attractive to

study within is reflective of a college that values the holistic

development of its students’ professional skills and both

reflects interest in and provides opportunity for the

development of some of the higher level skills in the affective

domain as well as skills related to the cognitive domain.

A college that offers extra-curricular activities opportunities,

such as sports, interest and cultural clubs provides an

environment of enrichment that leads to the students’ growth in

capabilities related to the affective domain, even though these

capabilities, by being extra-curricular, are not assessed directly

in any process or transcript of the college.

STAFF PROFILE

Academic staff are of crucial importance to the holistic effect

of the curriculum of a college. The academic staff are directly

contacting students for the purposes of providing instruction in

the content of the field and to model the practice of the

discipline and appropriate behaviours associated with

practising in the field. Therefore the knowledge and

capabilities of the staff are important.

Issues to be investigated as one considers the appropriateness

of the staff include:

the staff to student ratio;

the non-teaching duties of staff in the college;

the highest qualification of each staff member, and the

issuing authority of the qualification;

the work experience of staff members prior to entering the

college;

staff turn-over;

average duration of service in the college; and

language of instruction and other languages of staff and

students.

The staff to student ratio is important because it reflects the

size of classes. Where classes are large the teaching

methodology tends to reduce to telling by the teacher. Where

the class is small there can be discussion style interaction

between the students and the staff member. In discussion the

student is forced to be more actively engaged in the learning

process, and the teacher can perform on-going formative

assessment allowing the student to gain feedback concerning

performance and quality of knowledge and work. A staff to

student ratio that is small enough permits students the

opportunity for, but does not guarantee, that the teaching

activities will provide students with higher attainment in both

the cognitive and affective domains.

The non-teaching duties of staff affect the amount of time staff

have to deal with student based issues and also the quality of

teaching activities. Other duties affect teaching by both

impacting the time available to the staff member to perform

work related to teaching and also may act as a distraction from

attending to the needs of students.

Staff qualifications are important because each level of

qualification reflects a kind of learning that the staff member

has successfully engaged upon. This reflects a general level of

competence of the person to be able to enter and complete the

particular award and also reflects the kind of academic

attainment that the staff member is capable of achieving. Thus,

a person only holding a Bachelor or a coursework Master

award may have learnt information, but may feel difficulty in

coping with the potential challenges that students may present

through questioning, and so may act in a manner that

discourages questioning. Conversely a person with a research

award has probably developed the ability to work in poorly

defined problem spaces and probably enjoys the intellectual

stimulation of questions, and so may encourage students to ask

questions. In turn this characteristic of the staff member leads

to a different perception in the student as to whether the

discipline is a space to be explored or content to be learned.

Engineering is a field of activity involving practice to make

real things happen in the world. Therefore it is very beneficial

for the teachers in engineering colleges to have significant

industrial experience prior to entering academic work. A

teacher with experience can both provide case studies based in

their own experience and a developed perspective on the

profession that involves understanding how the profession

works in real industrial employment and the pressures under

which engineers work. This experience will provide the staff

member with the capacity to provide students with good

general advice about how to practice the profession and how to

work. If the majority of staff have very little work experience

outside the academic sector there are warning signs concerning

educational quality. If the staff do not have industrial

experience to bring to the classroom to enhance their manner of

teaching the curriculum is likely to shift in the direction of pure

theory without the realism introduced by the problems of

practice. If the staff salary rates are not competitive with the

local industry the lack of outside employment may be

indicative of recruitment of staff who were not able to find

industrial employment. This is not good for education in any

industrially applied field.

Staff turnover is an issue in most kinds of employment. In

education the turnover rate is significant because it may reflect

on the general college environment and also because of the

state of stability or otherwise that it indicates in the educational

environment. Where staff continue in employment for too long

they may become too settled in their ways and curriculum may

become bound by college traditions. It is generally helpful to a

college to have some turnover of staff. However, a high rate of

turnover is indicative of other problems. These problems may

concern the basic conditions of employment, such as salary

rates compared with local industries or some unpleasant

friction within the staff body, or a general sense of insecurity

about the future of the college. These things, of which a high

staff turnover may be symptoms, are indicators of problems in

the holistic curricular environment of the college.

Where there is a high staff turnover the college may also suffer

from a lack of continuity in style and form of academic work,

leading to students potentially confronting many changes of

policy and expectation, leading to academic confusion. Further,

the frequent changes of staff, especially if not managed to

prevent departures except at acceptable dates in the academic

calendar may result in disruption to students.

Many colleges claim to use English as a medium of instruction.

However, in some cases graduates of such colleges have very

poor English, which is inadequately developed for academic

purposes. An issue to investigate is the ethnicity of the student

and staff cohorts. Where staff and students share the same

ethnic language it is plausible, and seems likely from

observation of some cases, that although the official medium of

instruction is English there may not be forcing of students to

use English to ask questions to clarify points. The result is that

students having difficulty and needing to ask questions to

clarify points may ask in the shared non-English language and

receive an answer in that same language. The result is that the

real teaching and learning occurs in the ethnic language, rather

than the official medium. Such students are very poorly

prepared for entry into an Australian university where it is most

unlikely that the student will have instructors capable of

speaking their own language, and it is very likely that the

student will confront instructors who use English as a second

language, and therefore present a further layer of difficulty in

understanding.

The matters of staffing affect the outcomes achievable in the

cognitive domain, and particularly the more complex issues in

the cognitive domain that relate to the industrial use of

engineering learning.

EVALUATION FOR ADVANCED STANDING

The Australian education market has established a practice of

recognizing diploma awards from a diversity of overseas

colleges for the purpose of advanced standing entry into

engineering degrees. The simplest means to evaluate the

diploma for the purpose of advanced standing is to review

documentation provided by the college concerning the content

of its courses. Where there is a match in course content it is

normal to provide advanced standing credit for a course in the

degree award. This approach is poor practice because it does

not reflect that education and curriculum are holistic matters

pertaining to the development of the whole person to be

suitable to practice a profession. Thus, out understanding of the

purpose of engineering education must be broadened to

become:

to teach content necessary for the analysis and design of

engineering products;

to develop engineering methods of perceiving things and

the world; and

to acculturate the young engineer into the values and

practices of the profession.

Therefore any assessment of an engineering program as a

program of instruction to develop the student into an engineer

cannot be complete if only the substantive content to the list of

topics is considered. The more important issue is the broad

development of the student into the kind of person who can

practice engineering effectively. Therefore a broad ranging,

holistic assessment of the college is required.

This presents a new problem. There are so many colleges

offering diplomas and seeking recognition for advanced

standing entry in many universities that it is impractical and

inefficient for each university to perform a rounded assessment

of each college.

What would be useful would be a multi-dimensional

accreditation process that, rather than awarding an

accreditation pass, or fail, for each applicant college would

provide a multi-dimensional measure of the performance of the

college on each of a range of dimensions addressing issues as

described above. In some cases quantitative measures may be

developed and helpful, but in all cases the accreditation should

contain a descriptive element for the guidance of university

staff making advanced standing guidelines decisions, either for

particular cases or in a generic manner for efficient screening

and offer issuing processes by specialists in university entry,

not engineering.

Such a multidimensional approach would greatly assist the

making of wise decisions about particular applicants

concerning how much advanced standing, and in which course

areas is appropriate so that each applicant can be granted an

accelerated start to their degree that will not compromise either

their probability of success in the degree nor their post-

graduation competences, which inturn affect career success.

CONCLUSIONS

These issues arising out of consideration of Bloom’s taxonomy

of educational outcomes provide a basis to assess engineering

diploma level programs as the basis for advanced standing

entry into university engineering programs. The concept of a

multi-dimensional engineering college accreditation process

that provides guidance for people constructing advanced

standing entry requirements for engineering degrees would be

most useful.

This paper proposes a new kind of accreditation for diploma

level colleges teaching in engineering disciplines. The major

difference between the process suggested and the usual kind of

accreditation of degrees, which functions as a “good enough”

or “not good enough” categorical distinction process, is that the

proposed process is intended to provide a meaningful

description of the characteristics of the college award without a

simplistic “good enough” or “not good enough” outcome, and

without the creation of a rank ordering of colleges for

“quality”, whatever that term may mean in the context of

something as diverse and multi-faceted as engineering

education. Instead the accreditation would provide descriptive

factors that enable judgements about the match between the

educational outcomes of the college and the suitability of the

graduates to participate in particular post-graduation activities,

such as further education. Such an accreditation scheme may

also be of use to employers seeking to understand the kind of

person a graduate of a particular college is likely to be, and

therefore the suitability of that person for the kind of work the

employer is seeking to have done. In their local context

employers use local knowledge for this purpose, but this kind

of accreditation would enable judgements to be made in

relation to non-local graduates.

REFERENCES

[1] W. S. Adams, “The analog computer laboratory in

engineering education,” IEEE transactions on education,

vol. 7, pp. 59-62, 1964.

[2] N. Balabanian, “Thoughts on engineering education,” IRE

transactions on education, vol. 4, pp. 6-8, 1961.

[3] H. R. Cohen, “A suggestion for a modern teaching

approach to the high school physics curriculum,” IRE


[4] D. M. Grimes, “Quantum theory in the electrical

engineering curriculum,” IEEE transactions on education,

vol. 8, pp. 50-56, 1965.

[5] M. S. Gupta, “What to teach: understanding, designing and

revising the curriculum,” IEEE transactions on education,

vol. 24, pp. 262-266, 1981.

[6] N. A. Hall, “Current development in patterns of graduate

study,” IRE transactions on education, vol. 2, pp. 43-49,

1959.

[7] T. J. Healy, “Magnetics in the modern curriculum,” IRE


[8] T. J. Healy, “The case for abolishing the senior year in

American undergraduate engineering education,” IEEE


[9] J. J. Jonsson, “Microprocessors (uP's) in electrical

engineering education survey,” IEEE transactions on

education, vol. 24, pp. 3-8, 1981.

[10] H. K. Kesavan and B. R. Myers, “System theory in a

unified curriculum,” IRE transactions on education, vol. 4,

pp. 102-110, 1961.

[11] F. A. Lindholm, “Unipolar transistors: some aspects

related to their inclusion in electronics curricula,” IEEE


[12] J. Lubelfeld, “Education in molecular electronics,” IRE


[13] J. Merrill, “The scientific manpower shortage and its

implications for the engineering technician,” IRE


[14] G. F. Paskusz and B. Bussell, “Circuit theory in a unified

curriculum,” IRE transactions on education, vol. 3, pp. 84-

88, 1960.

[15] L. E. Saline, “The technology of information systems -

another challenge for engineering education,” IRE


[16] S. Seshu, “The curriculum and the education of engineers,”

IEEE transactions on education, vol. 10, pp. 65-67, 1967.

[17] P. E. Shields, “The role of an AC network calculator in the

electrical engineering curriculum at The Pennsylvania

State University,” IRE transactions on education, vol. 4,

pp. 22-25, 1961.

[18] J. G. Truxal, “Electrical machinery in an electronics-

oriented curriculum,” IRE transactions on education, vol.

1, pp. 66-69, 1958.

[19] E. A. Walker, “The time is now,” IRE transactions on

education, vol. 2, pp. 2-6, 1959.

[20] A. B. Weaver, “Physics courses in the new Arizona

engineering curriculum,” IRE transactions on education,

vol. 2, pp. 117-120, 1959.

[21] M. Betts and R. Smith, “Patterns of provision,” in

Developing the credit-based modular curriculum in higher

education. London: Palmer Press, 1998, pp. 15-34.

[22] J. Biggs, Teaching for quality learning at university: what

the student does. Buckingham, UK: Society for Research

into Higher Education, Open University Press, 1999.

[23] L. Brady, “Evaluating curriculum: major concerns,” in

Curriculum development, 5th ed. Sydney: Prentice-Hall,

1995, pp. 245-266.

[24] S. Butterfield, Educational objectives and national

assessment. Buckingham, Philadelphia: Open University

Press, 1995.

[25] S. Grundy, Curriculum: product or praxis. London, New

York, Philadelphia: The Fulmer Press, 1987.

[26] P. B. Joseph, “Understanding curriculum as culture,” in

Cultures of curriculum, P. B. Joseph, S. L. Braumann, M.

A. Windschill, E. R. Mikel, and N. S. Green, Eds. Matwah,

New Jersey: Lawrence Erlbaum Associates, 2000, pp. 15-

27.

[27] K. J. Kennedy, “Developing a curriculum guarantee for

overseas students,” Higher education research and

development, vol. 14, pp. 35-46, 1995.

[28] J. L. Ratcliff, “What is a curriculum and what should it

be?,” in Handbook of undergraduate curriculum: a

comprehensive guide to purposes, structures, practices and

change, J. G. Gaff, J. L. Ratcliff, and associates, Eds. San

Francisco: Jossey-Bass, 1997, pp. 5-29.

[29] S. Toohey, “Beliefs values and ideologies in course

design,” in Designing courses for higher education.

Buckingham, UK: Society for Research into Higher

Education, Open University Press, 1999, pp. 44-69.

[30] S. Amirtham, “Towards a contextualised curriculum: an

Indian report,” Ministerial formation, vol. 34, pp. 29-32,

1986.

[31] M. Roemmele, “Curriculum design for TEE,” in Voice of

the church in Asia, B. R. Ro, Ed. Singapore: Asia

Theological Association, 1975, pp. 23-35.

[32] N. Wran, M. Edwards, R. G. Gregory, and M. Gallagher,

Report of the committee on higher education funding.

Canberra: Australian Government Publishing Service,

1988.

[33] J. S. Dawkins, Higher education a policy statement.

Canberra: Australian Government Publishing Service,

1988.

[34] T. L. J. Ferris, “A technician is half an engineer - overseas

students and advanced standing in engineering degrees,”

presented at Proceedings of the 14th annual conference

Australasian Association for Engineering Education and

9th Australasian women in engineering forum engineering

education for a sustainable future, RMIT University,

Victoria, 2003.

[35] B. S. Bloom, M. D. Engelhart, E. J. Furst, W. H. Hill, and

D. R. Krathwohl, Taxonomy of educational objectives the

classification of educational goals handbook I: cognitive

domain. London: Longman Group Ltd, 1979.

[36] D. R. Krathwohl, B. S. Bloom, and B. B. Masia, Taxonomy

of educational objectives the classification of educational

goals handbook II: affective domain. London: Longman

Group Ltd, 1973.

[37] B. S. Bloom, “Reflections on the development and use of

the taxonomy,” in Bloom's taxonomy a forty-year

retrospective, vol. 93, part 2, Yearbook of the national

society for the study of education, L. W. Anderson and L.

A. Sosniak, Eds. Chicago, Illinois: The University of

Chicago Press, 1994, pp. 1-8.

[38] T. L. J. Ferris, “Consistency of student performance

throughout Bachelor of Engineering (Electronic

Engineering),” presented at Proceedings of the 9th annual

convention and conference putting students back into

engineering education, University of Ballarat, Victoria,

1997.

[39] J. Biggs, “Teaching across and within cultures: the issue of

international students,” in Learning and teaching in higher

education: advancing international perspectives, R.

Murray-Harvey and H. C. Silins, Eds., 1997, pp. 1-22.

[40] W. E. Deming, Out of the crisis: quality, productivity, and

competitive position. Cambridge, Cambridgeshire:

Cambridge University Press, 1986.

Approach for Evaluation of a College and Diploma Program for Advanced Standing Entry to a Degree...

Documents

Transcript of Approach for Evaluation of a College and Diploma Program for Advanced Standing Entry to a Degree...