Post on 03-Apr-2023
Exploring Innovation in Education and Research ©iCEER-2005
Tainan, Taiwan, 1-5 March 2005
Approach for Evaluation of a College and Diploma Program for Advanced Standing
Entry to a Degree Program
Timothy L.J. Ferris
timferris@ieee.org
Systems Engineering and Evaluation Centre, University of South Australia, Mawson Lakes, South Australia,
Australia
ABSTRACT: Recently there has been a shift in the educational background of overseas applicants for entry to Bachelor of
Engineering awards in Australian Universities. In addition, the 1988 Dawkins Higher Education policy proposed that Australian
higher education in general should be structured to enable individual people to progressively upgrade qualifications throughout their
career, with minimal loss of time compared with choosing a minimum time pathway to their ultimate educational goal from the
outset. These background factors have resulted in Australian universities being confronted with a need to assess holders of various
Diplomas in engineering disciplines with a view to granting entry to a Bachelor of Engineering award accredited by Institution of
Engineers Australia and consequently throughout the Washington Accord countries as a first award in engineering, with substantial
advanced standing based on the preceding studies. The numbers of applicants and the contextual factors result in a need for
Australian universities to establish advanced standing agreements with diverse ‘feeder colleges’ to the effect that a holder of a
particular diploma with a certain standard of attainment will be granted a certain advanced standing. Particularly where the ‘feeder
college’ is overseas there are complications to the process related to distance in both geographic and cultural dimensions. The author
has been involved in a task requiring determination of a generalisable set of criteria by which to assess the quality of education
achieved in a program of study involving a process to measure student attainment against a defensible standard, in this case Bloom’s
taxonomy of educational objectives, college academic staffing and facilities, in addition to the details of course and program content.
The discussion of the paper will provide basis for people concerned with: 1. Accreditation of programs for recognition as
engineering awards; 2. Recognition of advanced standing entry to accredited programs; and 3. Educational development assistance.
INTRODUCTION
Australian universities have received a flow of overseas
students over a very considerable period. During the history of
overseas students entering Australia there has been a series of
changes affecting the kind of people seeking to come to
Australia to study and the motivations prompting their
movement. This paper addresses certain issues arising from
some of the more recent changes in the overseas student cohort
coming to Australia and opens more general discussion of
issues arising from the point of view of curriculum, understood
in a broad sense as the entire set of educational activities and
arrangements established, in their context that lead to student
learning.
The changing context of overseas countries from which
overseas students come and changes in the Australian
educational system have resulted in many students studying in
their home country to attain a sub-degree level qualification in
a field related to a Bachelor award in engineering. Such
students then seek, and are, subject to various conditions
granted advanced standing that significantly reduces the
amount of study that they need to complete in Australia in
order to fulfil the requirements of a Bachelor of Engineering
award. This phenomenon requires that Australian universities
develop means to assess the quality of the various colleges and
their diploma awards in order to assess the implied equivalence
to parts of the universities’ own bachelor awards. The
importance of this issue has grown with the combination of
increasing absolute numbers of applicants and the fact that the
predominant academic pathway sought by intending overseas
students of engineering in Australia involves this two stage
process.
The issues discussed in the paper are important for the
consideration of the matter of accreditation of engineering
awards because the issues open significant issues concerning
the nature of engineering education and the qualities that a
person must develop in order to be an effective professional
engineer.
CURRICULUM
One, simple, view of curriculum sees curriculum as concerning
the itemisation of the substantive content, or topic areas, to be
taught through a program of studies. Such a simple view of
curriculum is common in many places, with engineering
academics being an example of a large body of participants in
education who hold to the simple view. Examples of such
usage of ‘curriculum’ by engineering academics can be seen in
the following references amongst many others that could also
have been cited [1-20]. In various ways these authors reflect a
view of curriculum as concerning the topic areas to be covered
by a program of study in order that that program of study will
lead to its graduates being qualified appropriately for a certain
field of activity. The emphasis of such work is on what the
student must learn about in order to be worthy of graduating in
the nominated field. However, there is much more to being
competent to perform significant work in a field than just
having knowledge of a certain set of topics.
However, one should note that the authors cited above come
from the USA, and date between 1955 and 1981. This range of
dates is important for two reasons, it pre-dates the diffusion of
more modern, and holistic, concepts of ‘curriculum’ and
because the background of the majority student group was
relatively homogeneous. The importance of the homogeneity of
the background of students is significant in the matter of the
holistic interpretation of curriculum because where the
background of students is homogeneous it is possible for the
educator to make considerable assumptions about the pre-
existing knowledge of the student that are reasonable, and are
consistent across the class. Any assumption that is consistent
across the class permits the educator to take that particular
matter of competence as given, rather than as a matter that
must be supported by deliberate educational action.
Consequently, in situations of homogeneous cultures, in which
all, or almost all, students bring very similar background
experience and knowledge to their studies it is practical for the
educator to assume the student background and to omit
processes for the development of such skills from the goals of
the educational process conducted.
In the more recent educational literature ‘curriculum’ has been
presented as concerning the holistic presentation of learning
experiences and context to the student [21-29]. Curriculum
includes the content of the material presented to the student, as
in the earlier conception, but also includes the manner in which
it is presented to the student, the learning tasks the student is
expected to perform in the process of the teaching and learning
engagement with the material, the educational institution
context including the physical and intangible resources
available to students, the facilities available for ‘extra
curricular’ activities such as clubs and societies, and the social
milieu created by the other students. The argument to support
adoption of this broad interpretation of curriculum is that the
presence and effectiveness of these factors affects the learning
that the student achieves through their program of study.
To some extent there has long been an intuitive sense that these
‘extra’ things are significant, with the response being the
establishment and strength of schools of the form of the
English ‘public schools’, which in addition to a reputation for
good academic standards, in the narrow sense above, also
provided opportunities for students have a broad range of
experiences that lead to development of the whole person in the
context of learning about particular academic content. The
intuitive sense concerning the value of a holistic, broad,
curriculum has been demonstrated through the willingness of
generations of parents to make significant sacrifices to send
their children to such schools in the expectation and belief that
their children would be developed in a broader range of ways
through that experience than they would have been had they
been sent to a school offering only a required content based
curriculum.
In post secondary education the appreciation of the holistic
purpose of education, being to develop the person to be a
person able to exercise certain kinds of skills and to bear
certain kinds of responsibility related to the exercise of those
skills seems to be little appreciated. The two major situations in
which the holistic purpose of professional education is clearly
acknowledged and the consequences of that acknowledgement
are worked through into practical expression in the practice of
the education are in military academies and theological
education. In both these fields, as different as they sound, there
is a need to develop students so that as graduates they will be
able to make decisions significantly impacting others. The
graduates must exercise considerable personal maturity and
judgement at very early career stages. In theological education
this is reflected through the acknowledgement of the
curriculum as requiring broad ranging development of the
student to develop the student in knowledge and personal
qualities to be suitable to practice ministry [30, 31]. In the
military academy case the graduates will make life and death
decisions affecting their own military unit and with potential to
calm or inflame passions influencing international politics.
Although engineering does not have such stringent personal
requirements of its practitioners it is appropriate for
engineering education to use a broad understanding of
curriculum because successful practice in engineering is
dependent upon a combination of knowledge, practice of
personal responsibility and possession of a broad range of
skills such as effective inter-personal and communication
skills. A narrow view of curriculum sees education in each of
these matters being achieved by means of courses focused on
each of the particular matters, whereas a holistic view of
curriculum sees many of the desirable personal qualities being
developed through the means by which other parts of the
curriculum are taught. Thus a holistic view of curriculum sees
that both the content and the means and context of the teaching
are important to the educational outcomes and some desirable
outcomes are understood as being achieved through the means
of teaching and assessing other desirable outcomes.
DAWKINS REFORMS IN AUSTRALIA
In Australia during the 1980’s there was political pressure
concerning the cost and status of post secondary education. In
the Australian education the then major supplier of funds for
post secondary education was the government sector, with the
federal government responsible for the provision of the funds
expended through tagged grants to the states even where the
states had formal authority to provide for the education. The
result was that funding for post secondary education was seen
as a federal issue.
Australia was in a period of restructuring of the economy
during this decade to effect a transition from the former
somewhat insular, local market oriented economy with a cost-
plus mentality affecting productivity, efficiency and
competitiveness. The result was that issues of efficiency in the
provision of educational services came to be emphasised in a
new way. A further factor was that the Australian Labour
Party, ALP, formed the government at the federal level from
1983 to 1996, and the traditional power base of the ALP is the
working person and other socially disadvantaged groups.
The combination of these factors led the federal government to
look for means to change the post secondary education sector
to provide pathways to provide means for individuals with
certain qualifications and work experience which suited their
achievements and intentions at some earlier time in their life to
upgrade their qualifications conveniently and with little or no
disadvantage when their circumstances, such as employment,
made it advantageous for them to make such an upgrade. The
first relevant government publication was a study discussing
various issues associated with the efficient and effective
delivery of education and training for the strengthening of the
national competitive position [32]. Amongst the
recommendations was that post secondary education should be
organised as a series of qualifications that can be used in a
progressive upgrading process through establishing all post
secondary qualifications as part of a layered framework, where
for example a trade certificate or diploma can function as a first
step towards a higher qualification, such as a degree, without a
repetition of study time resulting in the person expending an
increased aggregate duration in study to achieve the intended
higher award.
Following this publication the official policy document was
released [33]. The Dawkins policy made many significant
changes to higher education in Australia. To further efficiency
in the sector the system would be unified into a single higher
education sector with all institutions treated on the same basis
as universities rather than the preceding three tier system of
universities, institutes of technology and colleges of advanced
education, each specialising in certain parts of the educational
and training task and funded in significantly different ways.
The former system resulted in institutions which were
perceived as significantly different in status. One objective was
to flatten the perceived difference of status. However, in the
new system, there rapidly came to be a perceived ranking of
universities following from the status of the antecedent
institutions, which in the case of most of the newer universities
was some combination of antecedents which had been
amalgamated. The old Technical and Further Education,
TAFE, sector which had provided various certificate, trade and
technician type awards continued under the Dawkins policy
with the major change being the introduction of a unified
national curriculum centred on competency based assessment,
and thus permitting students to complete or upgrade awards
whilst making career moves around the country.
The policy also, and partly through the flattening of the system
which often linked providers of lower qualifications with
providers of degree qualifications within one institution,
required that the concept of multiple pathways to degrees be
created. This contrasted with the previous thinking. Previously
there was a profound separation of people at the time of
completion of secondary school. Some progressed into trade
and technician level qualifications, usually based on finishing
school before completion of year 12. These people were then
‘locked-in’ to a career path that was based on that choice, and
would normally remain in the lower employment ranks. The
others were those who completed year 12 at school and
proceeded to degree level study and would pursue a career path
in the so-called ‘professional’ ranks. Where a holder of a
TAFE award sought entry to a university program the TAFE
award was generally assessed from the point of view of
equivalence to year 12, rather than as providing equivalence to
some part of the degree award.
The effect was a strong discouragement for TAFE award
holders from seeking to upgrade qualifications because of the
time cost brought through the lack of recognition of previous
awards by the university sector. The government saw this as
discriminatory, because social class and area of upbringing
were significantly linked to entry to university programs, and
thus against the social pillar of ALP policy, and inefficient in
the use of educational resources or the ‘artificial’ restriction on
the progress of people to higher productivity occupations, and
thus against the concern for national competitiveness.
The policy requirement for alternative pathways to degrees has
required that universities recognise lower awards as providing
equivalence of at least part of the Bachelor award offered by
the university, and so to provide substantial advanced standing
for studies performed outside the university as a step towards
the completion of the requirements for the degree. The policy
is clear in indicating that the amount of advanced standing
should result in students pursuing a pathway in the same field
of endeavour progressing through their pathway in the same
time as it would have taken them pursuing the traditional
‘standard’ pathway. The shift in the TAFE sector to
competency based assessment created considerable controversy
in the higher education sector, which would received TAFE
graduates seeking to upgrade their qualifications, because of
the profoundly different basis of assessment in the two sectors
and the related issues of the kind of capability developed in
each place.
PROBLEMS OF ADVANCED STANDING ENTRY
Advanced standing entry into an engineering degree based on a
completed technician diploma award presents a number of
problems. The author has discussed these problems in [34]
finding that the challenges presented are several and complex.
The first problem is that if the diploma award is well designed
as a qualification to make the graduates effective in a particular
kind of employment, that is, is designed to educate technicians
appropriately to their role as technicians, then it is not simply
the first half of an engineering degree. The role of technicians
in industry is to perform a range of reasonably routine or work
as directed activities that require a considerable level of
technical skill. Thus technicians are often used to perform
routine calculations and modelling of proposed products, and
to perform the hands-on work of dealing with the stuff of the
discipline in the workplace. In contrast, the purpose of the
engineer in the workplace is to have an elevated level of
analytical knowledge of the things related to the technical
discipline, and consequently the ability to make judgements
about the most appropriate solutions to particular technical
issues when selecting between a range of candidate solutions.
This role requires that the engineer develop different
capabilities through the process of professional formation.
The different capabilities required of engineers involve the
engineer needing to develop an elevated level of analytical
skill, in order to be able to do the more complex levels of
technical analysis including the relating of observations of real
states of things to the analysis of those things. The observation
of states of things may be found in either the quantitative
observation process of measurement, in which case the relation
of the observed to conclusions can often be made using
formulae, and more complexly, in the qualitative observation
of things and the recognition and drawing conclusions of
significant factors that can be found in the observations. This is
a difficult process to perform and relies on the ability of the
engineer to draw insight from observations so that appropriate
action results. As a result the education of engineers
emphasises the development of high capability in analysis of
information, the drawing of insight and the making of
decisions.
In contrast, since many employers use technicians in roles
involving close hands-on contact with equipment the training
programs need to emphasize skills of handling and working
with equipment, in the construction, testing and maintenance of
that equipment. Emphasis on processes such as troubleshooting
and repair and the trade related skills of working with the
hardware is proper to the technician kind of work, but is
omitted in most engineer education programs because the
contribution of the engineer in industry is different than that of
the technician.
The second problem is that if the diploma is designed as the
beginning of a degree the emphasis in the teaching of the
diploma will be on the subject matter that is usually taught at
the beginning of an engineering degree so that the diploma will
provide the student with a relatively smooth transition in to
their later degree level studies. In this case the emphasis in the
design of the diploma is on the desired final destination of the
more able students in the class, rather than on providing a
coherent terminal award that serves the students who do not
continue and their employers well through provision of useful
and identifiable skills at the end of the diploma.
However, there is another force at work, most clearly seen in
the case of colleges in countries outside Australia which feed a
large number of overseas students to Australia. In some of
these countries the demand for post-secondary education vastly
exceeds the number of places available in universities for
degree level study and consequently there has been a spawning
of diploma level colleges. In certain countries many of the
colleges are operated by private sector businesses, in contrast
to the government operated universities, with the result that the
college proprietors’ objectives may be skewed by the business
imperative to generate profit. In the marketing performed by
many of these colleges one major emphasis, both to assert the
quality of their college and also to pursue the interest in
potential students to obtain a degree by some means, is the
foreign universities that provide substantial advanced standing,
through formal agreements, to diploma graduates of the
college. In the extreme this has caused difficulties for the
universities so named because at times the college marketing
has provided the market with the impression that the advanced
standing is either more, or with less conditions attached, than
the university has agreed. In some cases the university has
received negative publicity resulting from the disputes arising.
Another difficulty that this educational marketplace has
generated is that it has established the diploma plus advanced
standing entry to a degree as a ‘normal’ pathway for a school-
leaver to become an engineer. The perceived ‘normality’ of this
two stage pathway to a degree appears to have resulted in
distortion of the diploma awards of the colleges to satisfy the
‘upgrade’ market at the expense of not providing a diploma
award that is well adjusted to serve the employment interests
and employers of those who complete their studies at the
diploma level. However, the recognised lower desirability of
this process compared with direct entry to an engineering
degree result in the colleges accepting a class cohort who have
achieved at a standard lower than required for university entry.
The lower entry standard demands evaluation of the fine details
of the college level education or training from a holistic
curriculum point of view rather than the relatively easy course
content based approach. The need for holistic review of
curriculum has been found as a result of the difficulty that
some students who have transferred for advanced standing
entry have experienced. Therefore as a matter of ethical
concern for the negative effects of providing too much
advanced standing it is necessary to evaluate the underlying
basis on which such decisions are made. Bloom’s taxonomy
[35, 36] of educational objectives.
BLOOM’S TAXONOMY OF EDUCATIONAL OBJECTIVES
Bloom’s taxonomy of educational objectives [35, 36] was
introduced in 1954 and rapidly found broad acceptance as a
foundation for enabling program and assessment development
thinking by educators [37]. The taxonomy was rapidly
accepted in many places because it provided an effective
means to discuss the diversity of kinds of education objective
that could be obtained in various circumstances and it provided
a sensible basis for planning of teaching and assessment
activities to match the purpose of the particular educational
activity.
The taxonomy divides educational objectives into three
domains, the cognitive, the affective and the psycho-motor
skills. The work of the original group led by Bloom addressed
the former two domains, omitting the latter as a field not
touched. In each domain the possible kinds of capability that
could be achieved are described as a hierarchy of five or six,
depending on the domain, levels of possible attainment. Most
of the levels are further subdivided into several classes of
relevant factors, which are regarded as being different but at a
similar level of complexity and hence equivalent in the
hierarchy.
The Cognitive Domain is divided:
1. Knowledge
o Knowledge of Specifics
o Knowledge of the Ways of Dealing with Specifics
o Knowledge of the Universals and Abstractions in a Field
2. Comprehension
o Translation
o Interpretation
o Extrapolation
3. Application
4. Analysis
o Analysis of Elements
o Analysis of Relationships
o Analysis of Organizational Principles
5. Synthesis
o Production of a Unique Communication
o Production of a Plan, or Proposed Set of Operations
o Derivation of a Set of Abstract Relations
6. Evaluation
o Judgements in Terms of Internal Evidence
o Judgements in terms of External Criteria
The Affective Domain is divided:
1. Receiving
o Awareness
o Willingness to Receive
o Controlled or Selected Attention
2. Responding
o Acquiescence in Responding
o Willingness to Respond
o Satisfaction in Response
3. Valuing
o Acceptance of a Value
o Preference for a Value
o Commitment
4. Organization
o Conceptualization of a Value
o Organization of a Value System
5. Characterization by a Value Complex
o Generalized Set
o Characterization
The two domains presented in this form show the idea of a
hierarchy of complexity of outcomes in each domain. Further,
it is clear that educational activity could be structured in such a
way as to result in capabilities described by any of these
possible outcomes and therefore an assessment of an
educational structure could be made in terms of the typical
achievement of typical graduates of the program with respect
to the hierarchy. In addition, the relation of elements in the
hierarchy to the capabilities required for particular kinds of
work can be seen, with the result that where an educational
program exists to support some kind of industrial need for
training and education the taxonomy presents a reasonable
analysis against which to compare the program outcomes.
BLOOM’S TAXONOMY AS A MEASURE OF CURRICULUM
Where a decision must be made to grant, or not grant,
advanced standing to an applicant for a Bachelor of
Engineering on the basis of a diploma in a related field there
must be some sound basis for the decision. The dilemma is that
the student who would duplicate their previous study in the
later award will feel at least some measure of frustration over
any duplication, especially when the tuition fees and living
expenses associated with that additional study time required are
substantial, as they are for overseas students studying in
Australia. The educational problem is that the duplication may
be perceived by the student as the student sees similar content
covered but not be accepted by the faculty on the grounds that
the non-content learning that the student is supposed to develop
through the teaching and learning experience of the course is
different. On the other side of the dilemma is the problem that
the student granted advanced standing will enter a higher level
course to share with classmates who have learned similar
content, the usual basis for advanced standing judgements, but
by a different means that results in instructor assumptions
about overall student competence moulded by the majority
cohort, those going straight through, providing a mismatch
with the competence of the advanced standing students. The
result can be extremely uncomfortable for the advanced
standing student, particularly as this student faces the
adjustment problem in a class context where the instructor does
not normally need to deal with student adjustment to a new
educational situation and is not as aware of the possible
problems for some students as is necessary to protect those
students from the risk of lack of success.
For this reason, the advanced standing question needs to
consider the holistic teaching and learning context of the
college in which the antecedent study was completed. The
holistic context includes the course content, the assessment
methods applied to the students, the college facilities, and the
staff profile. Investigation of these issues is more important in
the case of overseas colleges than for local TAFE colleges, in
the Australian context, because of the wide diversity of foreign
institutions, most of which do not have a track record known
immediately to the assessor and the diversity of educational
traditions and approaches used by the colleges, as compared
with the small number of colleges within Australia that feed
such applicants into degree level awards, all of which are
reasonably close to the experience of the assessor and all of
which are members of a known educations system.
The principle criterion for assessing the effectiveness of
diploma colleges in preparing candidates for advanced standing
entry to a degree award is to look at the kind of capability
created by the study performed at the diploma college, and to
compare this with what is achieved at various points through
the degree. The structure of Bloom’s taxonomy is a reasonable
approach to take to this assessment of the holistic development
in the student through the study of certain content.
PROGRAM CONTENT
To practice design engineering any person needs to have an
effective knowledge of a significantly broad range of technical
topics at a depth that the person is able to apply all the
knowledge possessed, know what ever else needs to be known
to complete the design and capable to learn what is needed, to
create a new design that will satisfy the need and the ability to
evaluate and criticise the design achieved in order to determine
appropriate modifications of the design. Mere knowledge of
the standard textbook presentation of the technical topics, and
the ability to perform well in examination of the student’s
ability to know and describe that content is insufficient to
determine that the student is competent to practice as an
engineer. The lack of correlation between success in
coursework courses and project courses within an engineering
degree is an aspect of the phenomenon described above [38].
Fields of engineering not directly related to the design of new
products require similar capabilities to know, use and apply
understanding of technical matters.
In the work concerning relative performance in coursework and
project courses [38] it was reported that students showed
significant differences in their average mark in all coursework
courses of the Bachelor of Engineering award compared with
the final year project courses of the award. This finding could
be criticized by questioning whether the expectation of
required performance on the part of the student to achieve a
certain mark was significantly different between the two
classes of course. There is no absolute means to eliminate the
potential problem caused by this criticism. However, in the
same work [38] it was found that students with an Australian
“permanent home address” in the University record system
tended to increase in performance, and so achieved a typical
result one grade, approximately 10%, better in the project
course compared with their own result in the coursework part
of the degree. In contrast, students with a “permanent home
address” outside Australia achieved a typical result one grade,
approximately 10%, lower in the project course compared with
their own result in the coursework part of the degree. The fact
that a bidirectional correlation of a factor to relative
coursework and project result could be identified is an
indicator that difference of gross assessment expectations
between the kinds of course was not the dominant factor.
The particular correlation of “permanent home address” and
relative performance in particular kinds of course raises the
question of possible racist bias. The principle defence against
this question is that a student’s status as an overseas student is
not obvious to academic staff on any documentation used in the
assessment process, and secondly, that by appearance and
ethnicity the diversity of the Australian population and
Australian resident and citizen student cohort is difficult to
distinguish from the overseas student cohort.
Consequently the correlation of results to “permanent home
address” must be related to the difference in the way in which
the students satisfy the expectations of academic staff for
assessment of their work with a certain grade. Clearly the
overseas students are more able to match the expectations
when performing coursework courses than when those same
students are performing the relatively open-ended project
course. This is, following Biggs [39], suggestive of two factors:
1. That overseas students have experienced a learning culture
resulting in different competence to deal with open-ended
project type problems; and
2. That the manner of education in the courses that the
overseas students had actually studied whilst students did
not motivate and enable them to approach the open-ended
project situation effectively.
Biggs argues that, contrary to a popular stereotype of Asian
origin students, such students regard western education as
valuable, and so are willing to make major sacrifices in order to
obtain such education. In addition, Biggs notes that whilst such
students often have difficulty learning to deal with the open-
endedness and uncertainty produced by the Australian teaching
styles and expectations placed upon students, once the
difficulties have been overcome, the Asian students often
become excellent performers in such ill-defined teaching and
learning situations, and not infrequently are represented among
the top performers.
The curriculum issue clearly concerns not only the content,
information, that is taught and learned but also the personal and
professional qualities developed in the student as a result of the
educational experience and environment. Therefore, whilst the
fact that the content of engineering education must address an
appropriate range of engineering discipline knowledge to
enable service within one of the fields of engineering, the
manner in which that content is taught and the kind of
capability with respect to Bloom’s taxonomy of educational
outcomes that is achieved must be considered.
In much of engineering education the emphasis is on the
attainment of cognitive qualities that map to levels 1 to 4 in the
Cognitive Domain of Bloom’s taxonomy. Thus students are
typically taught methods and content that enables the analysis
of phenomena and technology of interest to the particular
engineering discipline. Such knowledge is useful for people
working at the level of technical work as directed by a senior
engineering leader. In such a situation the major contributions
made to the product architecture and technical strategy are
made by the technical leader and the ‘engineers’ do analysis
and other technical work associated with investigation of the
hypotheticals that the technical leader has proposed. However,
‘engineers’ whose skills are limited to the Bloom cognitive
domain levels 1 to 4 are not able to confidently and effectively
propose new product ideas and techniques and perform all of
the tasks required to bring those ideas to fruition.
The special gap in the cognitive abilities of such people
pertains to levels 5 and 6 of the hierarchy. The additional
capabilities require skills in synthesis, the construction of new
sets of ideas that are novel, unprecedented in the experience of
the person, and the skill of evaluation, the ability to effectively
and efficiently criticize one’s own proposal, or that of another
person in relation to the appropriateness of the idea for a
particular application. The primary expression of synthesis in
engineering is in the construction of sets of ideas associated
with actual or possible things and the evaluation concerns the
critical testing and evaluation of those things to determine that
they are suitable solutions to the problem at hand.
In the analysis of a college it is necessary to delve into the
description of the curriculum to determine what is meant by the
headings of topics covered to determine how those topics relate
to the development of the higher order skills required for an
effective engineer.
ASSESSMENT
It is well-known that people respond to any incentive scheme
that is offered to them by modification of behaviour in order to
maximise return for effort. This issue is a potential problem
with respect to quality systems in production [40]. In education
the major incentive scheme offered to students is presented via
assessment, and is the gating of students as eligible to proceed
to the next stage of education or career. Since assessment has
the gating function for the possibility of the student progressing
to the next stage of education students do what they believe is
required to achieve success according to their criteria. To the
normal student success is advancement to the next stage of
education. It is the unusual student who regards some other
objective as “success”. Such other objectives that may be
regraded by some as success include gaining a useful
knowledge of the subject area that enables the student to solve
some question that motivated their choice to study the course,
or to obtain satisfaction from developing a sense of gaining a
strong level of mastery of the subject matter.
In assessment of the effectiveness of a particular college
program in developing its graduates to be worthy of advanced
standing entry into a different educational program one must
work on the statistics of the typical member of the student
cohort, not on the characteristics of the unusual candidate.
Working with the statistically common outcome is a tractable
problem that can be found through evaluation of the
assessment processes of the subject college. The process
requires evaluation of:
1. The assessment exercises set by the academic staff. This
includes assignment sheets, examination question, practical
task sheets and other similar items.
2. Samples of the student responses to the assessment
exercises.
3. The results awarded by the examiner for the work reviewed
in 2 above.
Each of these items is important. The assessment exercises set
are important because they reflect the instructor’s expectation
concerning what students should be able to achieve as a result
of the instruction provided. In addition, the assessment
exercises reflect the perspective of the instructor concerning
what is truly important about the development of knowledge in
the discipline. The review must deduce the kind of skills tested
through the assessment processes according to the taxonomy of
those skills provided by Bloom’s taxonomy. Reasonable
assessment is likely to have elements of all the lower levels of
the taxonomy, because in order to perform any higher order
cognitive skill there is a need to have, for example relevant
knowledge. However, the important points are to note the
highest level of the taxonomy that is tested through the
questions set and the proportion of the assessment that relates
to each level of the hierarchy. In addition, it is useful to review
the balance of knowledge of specific details of the course
content and knowledge of the general principles of the course
that are required in order to succeed.
Samples of actual student assessment work are important
because they enable an assessment of the kind of work that
students are capable of producing as a result of their
educational experiences. This demonstrates quite plainly where
the educational process demonstrates effectiveness or
otherwise in the achievement of the goals implied by the
taxonomical levels of the assessment tasks.
Comparison of the work produced by students with the marks
awarded to the students for that work is important because this
shows the kind of capability that is rewarded with particular
graded results. This enables the evaluator to make judgements
about the competence of a student achieving a particular grade
from the college. This affects whether the quantity of advanced
standing awarded should be dependant on the level of pass
achieved.
COLLEGE FACILITIES
The college facilities and physical infrastructure are relevant to
the performance of student learning. The college facilities
provide the infrastructure of the learning environment.
The physical state of teaching spaces affects issues such as the
ability of students to see and hear the instructor during classes.
Therefore the teaching spaces should be sufficiently quiet and
well illuminated to provide for convenient and safe study. The
space available to students reflects the expectations concerning
factors such as the expectation of the number of reference
works that a student is expected to need to have open and
accessible whilst performing a study task.
Engineering study involves laboratory work. The amount,
quality and suitability of laboratory equipment reflects the
capability for the college to provide appropriate practical
learning experiences for students. However, in greater detail of
investigation one should determine whether the laboratory
equipment is designed only to provide examples of phenomena
operating normally, or with set piece faults, or whether the
equipment permits open development of practicals that enable
exploration of diverse effects. Generally, the greater the
flexibility of the laboratory equipment the more different
practical exercises that students can experience, and the greater
the opportunity provided for students to learn about the wide
range of things that may happen under diverse circumstances
and the subtlety of the learning that students may develop. In
electronics, for example, some laboratory equipment makers
produce systems that can be used for demonstration of a very
limited range of phenomena, and are generally quite expensive.
In contrast the laboratory could be established to use class time
assembled circuits built from basic components, and the capital
cost reduction may provide consumables for the component
based practical work for a considerable duration.
More broadly the college library reflects the learning
environment provided. Issues concern whether the library has
an adequate stock of books in the field around the areas of
interest of the students. College libraries reflect the resources
available to instructors for the preparation of their teaching
materials and to students in their addressing of assessment
questions and general learning opportunities. Where the library
is small and narrowly focused on providing access to textbooks
and other very similar material it is fair to understand the likely
educational outcomes falling short of the highest levels of the
cognitive domain hierarchy of outcomes. In addition, a broad
library reflects both the possibility for, and usually a college
environment in which there is encouragement for students to
explore issues around the subject matter of their program,
rather than to narrowly focus on the content of the courses of
instruction.
The physical environment of the college and its immediate
surrounds affects the desirability for students of staying in or
around the college throughout the day. If there are plenty of
spaces available and conducive to private study and small
group discussion the college provides an environment in which
the students may learn to work collaboratively and to use their
time efficiently. Such an environment assists students with
being able to complete study tasks using peer discussion as a
tool to assist learning, and also to learn to use inter-class breaks
efficiently for the conduct of study tasks. These skills will be
beneficial both after transfer to a university with advanced
standing entry, because these skills are typical of what is
expected and done in universities and in turn, these skills are
useful tools for career progress because collaborative and time
pressured work is typical of the workplace.
The college may provide the pleasant environment by either
establishing all the necessary facilities on its own real estate, or
by choosing a site where at least some of the facilities are
available in the surrounding community.
The establishment of a college environment that is attractive to
study within is reflective of a college that values the holistic
development of its students’ professional skills and both
reflects interest in and provides opportunity for the
development of some of the higher level skills in the affective
domain as well as skills related to the cognitive domain.
A college that offers extra-curricular activities opportunities,
such as sports, interest and cultural clubs provides an
environment of enrichment that leads to the students’ growth in
capabilities related to the affective domain, even though these
capabilities, by being extra-curricular, are not assessed directly
in any process or transcript of the college.
STAFF PROFILE
Academic staff are of crucial importance to the holistic effect
of the curriculum of a college. The academic staff are directly
contacting students for the purposes of providing instruction in
the content of the field and to model the practice of the
discipline and appropriate behaviours associated with
practising in the field. Therefore the knowledge and
capabilities of the staff are important.
Issues to be investigated as one considers the appropriateness
of the staff include:
the staff to student ratio;
the non-teaching duties of staff in the college;
the highest qualification of each staff member, and the
issuing authority of the qualification;
the work experience of staff members prior to entering the
college;
staff turn-over;
average duration of service in the college; and
language of instruction and other languages of staff and
students.
The staff to student ratio is important because it reflects the
size of classes. Where classes are large the teaching
methodology tends to reduce to telling by the teacher. Where
the class is small there can be discussion style interaction
between the students and the staff member. In discussion the
student is forced to be more actively engaged in the learning
process, and the teacher can perform on-going formative
assessment allowing the student to gain feedback concerning
performance and quality of knowledge and work. A staff to
student ratio that is small enough permits students the
opportunity for, but does not guarantee, that the teaching
activities will provide students with higher attainment in both
the cognitive and affective domains.
The non-teaching duties of staff affect the amount of time staff
have to deal with student based issues and also the quality of
teaching activities. Other duties affect teaching by both
impacting the time available to the staff member to perform
work related to teaching and also may act as a distraction from
attending to the needs of students.
Staff qualifications are important because each level of
qualification reflects a kind of learning that the staff member
has successfully engaged upon. This reflects a general level of
competence of the person to be able to enter and complete the
particular award and also reflects the kind of academic
attainment that the staff member is capable of achieving. Thus,
a person only holding a Bachelor or a coursework Master
award may have learnt information, but may feel difficulty in
coping with the potential challenges that students may present
through questioning, and so may act in a manner that
discourages questioning. Conversely a person with a research
award has probably developed the ability to work in poorly
defined problem spaces and probably enjoys the intellectual
stimulation of questions, and so may encourage students to ask
questions. In turn this characteristic of the staff member leads
to a different perception in the student as to whether the
discipline is a space to be explored or content to be learned.
Engineering is a field of activity involving practice to make
real things happen in the world. Therefore it is very beneficial
for the teachers in engineering colleges to have significant
industrial experience prior to entering academic work. A
teacher with experience can both provide case studies based in
their own experience and a developed perspective on the
profession that involves understanding how the profession
works in real industrial employment and the pressures under
which engineers work. This experience will provide the staff
member with the capacity to provide students with good
general advice about how to practice the profession and how to
work. If the majority of staff have very little work experience
outside the academic sector there are warning signs concerning
educational quality. If the staff do not have industrial
experience to bring to the classroom to enhance their manner of
teaching the curriculum is likely to shift in the direction of pure
theory without the realism introduced by the problems of
practice. If the staff salary rates are not competitive with the
local industry the lack of outside employment may be
indicative of recruitment of staff who were not able to find
industrial employment. This is not good for education in any
industrially applied field.
Staff turnover is an issue in most kinds of employment. In
education the turnover rate is significant because it may reflect
on the general college environment and also because of the
state of stability or otherwise that it indicates in the educational
environment. Where staff continue in employment for too long
they may become too settled in their ways and curriculum may
become bound by college traditions. It is generally helpful to a
college to have some turnover of staff. However, a high rate of
turnover is indicative of other problems. These problems may
concern the basic conditions of employment, such as salary
rates compared with local industries or some unpleasant
friction within the staff body, or a general sense of insecurity
about the future of the college. These things, of which a high
staff turnover may be symptoms, are indicators of problems in
the holistic curricular environment of the college.
Where there is a high staff turnover the college may also suffer
from a lack of continuity in style and form of academic work,
leading to students potentially confronting many changes of
policy and expectation, leading to academic confusion. Further,
the frequent changes of staff, especially if not managed to
prevent departures except at acceptable dates in the academic
calendar may result in disruption to students.
Many colleges claim to use English as a medium of instruction.
However, in some cases graduates of such colleges have very
poor English, which is inadequately developed for academic
purposes. An issue to investigate is the ethnicity of the student
and staff cohorts. Where staff and students share the same
ethnic language it is plausible, and seems likely from
observation of some cases, that although the official medium of
instruction is English there may not be forcing of students to
use English to ask questions to clarify points. The result is that
students having difficulty and needing to ask questions to
clarify points may ask in the shared non-English language and
receive an answer in that same language. The result is that the
real teaching and learning occurs in the ethnic language, rather
than the official medium. Such students are very poorly
prepared for entry into an Australian university where it is most
unlikely that the student will have instructors capable of
speaking their own language, and it is very likely that the
student will confront instructors who use English as a second
language, and therefore present a further layer of difficulty in
understanding.
The matters of staffing affect the outcomes achievable in the
cognitive domain, and particularly the more complex issues in
the cognitive domain that relate to the industrial use of
engineering learning.
EVALUATION FOR ADVANCED STANDING
The Australian education market has established a practice of
recognizing diploma awards from a diversity of overseas
colleges for the purpose of advanced standing entry into
engineering degrees. The simplest means to evaluate the
diploma for the purpose of advanced standing is to review
documentation provided by the college concerning the content
of its courses. Where there is a match in course content it is
normal to provide advanced standing credit for a course in the
degree award. This approach is poor practice because it does
not reflect that education and curriculum are holistic matters
pertaining to the development of the whole person to be
suitable to practice a profession. Thus, out understanding of the
purpose of engineering education must be broadened to
become:
to teach content necessary for the analysis and design of
engineering products;
to develop engineering methods of perceiving things and
the world; and
to acculturate the young engineer into the values and
practices of the profession.
Therefore any assessment of an engineering program as a
program of instruction to develop the student into an engineer
cannot be complete if only the substantive content to the list of
topics is considered. The more important issue is the broad
development of the student into the kind of person who can
practice engineering effectively. Therefore a broad ranging,
holistic assessment of the college is required.
This presents a new problem. There are so many colleges
offering diplomas and seeking recognition for advanced
standing entry in many universities that it is impractical and
inefficient for each university to perform a rounded assessment
of each college.
What would be useful would be a multi-dimensional
accreditation process that, rather than awarding an
accreditation pass, or fail, for each applicant college would
provide a multi-dimensional measure of the performance of the
college on each of a range of dimensions addressing issues as
described above. In some cases quantitative measures may be
developed and helpful, but in all cases the accreditation should
contain a descriptive element for the guidance of university
staff making advanced standing guidelines decisions, either for
particular cases or in a generic manner for efficient screening
and offer issuing processes by specialists in university entry,
not engineering.
Such a multidimensional approach would greatly assist the
making of wise decisions about particular applicants
concerning how much advanced standing, and in which course
areas is appropriate so that each applicant can be granted an
accelerated start to their degree that will not compromise either
their probability of success in the degree nor their post-
graduation competences, which inturn affect career success.
CONCLUSIONS
These issues arising out of consideration of Bloom’s taxonomy
of educational outcomes provide a basis to assess engineering
diploma level programs as the basis for advanced standing
entry into university engineering programs. The concept of a
multi-dimensional engineering college accreditation process
that provides guidance for people constructing advanced
standing entry requirements for engineering degrees would be
most useful.
This paper proposes a new kind of accreditation for diploma
level colleges teaching in engineering disciplines. The major
difference between the process suggested and the usual kind of
accreditation of degrees, which functions as a “good enough”
or “not good enough” categorical distinction process, is that the
proposed process is intended to provide a meaningful
description of the characteristics of the college award without a
simplistic “good enough” or “not good enough” outcome, and
without the creation of a rank ordering of colleges for
“quality”, whatever that term may mean in the context of
something as diverse and multi-faceted as engineering
education. Instead the accreditation would provide descriptive
factors that enable judgements about the match between the
educational outcomes of the college and the suitability of the
graduates to participate in particular post-graduation activities,
such as further education. Such an accreditation scheme may
also be of use to employers seeking to understand the kind of
person a graduate of a particular college is likely to be, and
therefore the suitability of that person for the kind of work the
employer is seeking to have done. In their local context
employers use local knowledge for this purpose, but this kind
of accreditation would enable judgements to be made in
relation to non-local graduates.
REFERENCES
[1] W. S. Adams, “The analog computer laboratory in
engineering education,” IEEE transactions on education,
vol. 7, pp. 59-62, 1964.
[2] N. Balabanian, “Thoughts on engineering education,” IRE
transactions on education, vol. 4, pp. 6-8, 1961.
[3] H. R. Cohen, “A suggestion for a modern teaching
approach to the high school physics curriculum,” IRE
transactions on education, vol. 2, pp. 69-72, 1959.
[4] D. M. Grimes, “Quantum theory in the electrical
engineering curriculum,” IEEE transactions on education,
vol. 8, pp. 50-56, 1965.
[5] M. S. Gupta, “What to teach: understanding, designing and
revising the curriculum,” IEEE transactions on education,
vol. 24, pp. 262-266, 1981.
[6] N. A. Hall, “Current development in patterns of graduate
study,” IRE transactions on education, vol. 2, pp. 43-49,
1959.
[7] T. J. Healy, “Magnetics in the modern curriculum,” IRE
transactions on education, vol. 5, pp. 172-175, 1962.
[8] T. J. Healy, “The case for abolishing the senior year in
American undergraduate engineering education,” IEEE
transactions on education, vol. 7, pp. 39-40, 1964.
[9] J. J. Jonsson, “Microprocessors (uP's) in electrical
engineering education survey,” IEEE transactions on
education, vol. 24, pp. 3-8, 1981.
[10] H. K. Kesavan and B. R. Myers, “System theory in a
unified curriculum,” IRE transactions on education, vol. 4,
pp. 102-110, 1961.
[11] F. A. Lindholm, “Unipolar transistors: some aspects
related to their inclusion in electronics curricula,” IEEE
transactions on education, vol. 11, pp. 25-33, 1968.
[12] J. Lubelfeld, “Education in molecular electronics,” IRE
transactions on education, vol. 5, pp. 136-139, 1962.
[13] J. Merrill, “The scientific manpower shortage and its
implications for the engineering technician,” IRE
transactions on education, vol. 3, pp. 3-10, 1960.
[14] G. F. Paskusz and B. Bussell, “Circuit theory in a unified
curriculum,” IRE transactions on education, vol. 3, pp. 84-
88, 1960.
[15] L. E. Saline, “The technology of information systems -
another challenge for engineering education,” IRE
transactions on education, vol. 4, pp. 91-97, 1961.
[16] S. Seshu, “The curriculum and the education of engineers,”
IEEE transactions on education, vol. 10, pp. 65-67, 1967.
[17] P. E. Shields, “The role of an AC network calculator in the
electrical engineering curriculum at The Pennsylvania
State University,” IRE transactions on education, vol. 4,
pp. 22-25, 1961.
[18] J. G. Truxal, “Electrical machinery in an electronics-
oriented curriculum,” IRE transactions on education, vol.
1, pp. 66-69, 1958.
[19] E. A. Walker, “The time is now,” IRE transactions on
education, vol. 2, pp. 2-6, 1959.
[20] A. B. Weaver, “Physics courses in the new Arizona
engineering curriculum,” IRE transactions on education,
vol. 2, pp. 117-120, 1959.
[21] M. Betts and R. Smith, “Patterns of provision,” in
Developing the credit-based modular curriculum in higher
education. London: Palmer Press, 1998, pp. 15-34.
[22] J. Biggs, Teaching for quality learning at university: what
the student does. Buckingham, UK: Society for Research
into Higher Education, Open University Press, 1999.
[23] L. Brady, “Evaluating curriculum: major concerns,” in
Curriculum development, 5th ed. Sydney: Prentice-Hall,
1995, pp. 245-266.
[24] S. Butterfield, Educational objectives and national
assessment. Buckingham, Philadelphia: Open University
Press, 1995.
[25] S. Grundy, Curriculum: product or praxis. London, New
York, Philadelphia: The Fulmer Press, 1987.
[26] P. B. Joseph, “Understanding curriculum as culture,” in
Cultures of curriculum, P. B. Joseph, S. L. Braumann, M.
A. Windschill, E. R. Mikel, and N. S. Green, Eds. Matwah,
New Jersey: Lawrence Erlbaum Associates, 2000, pp. 15-
27.
[27] K. J. Kennedy, “Developing a curriculum guarantee for
overseas students,” Higher education research and
development, vol. 14, pp. 35-46, 1995.
[28] J. L. Ratcliff, “What is a curriculum and what should it
be?,” in Handbook of undergraduate curriculum: a
comprehensive guide to purposes, structures, practices and
change, J. G. Gaff, J. L. Ratcliff, and associates, Eds. San
Francisco: Jossey-Bass, 1997, pp. 5-29.
[29] S. Toohey, “Beliefs values and ideologies in course
design,” in Designing courses for higher education.
Buckingham, UK: Society for Research into Higher
Education, Open University Press, 1999, pp. 44-69.
[30] S. Amirtham, “Towards a contextualised curriculum: an
Indian report,” Ministerial formation, vol. 34, pp. 29-32,
1986.
[31] M. Roemmele, “Curriculum design for TEE,” in Voice of
the church in Asia, B. R. Ro, Ed. Singapore: Asia
Theological Association, 1975, pp. 23-35.
[32] N. Wran, M. Edwards, R. G. Gregory, and M. Gallagher,
Report of the committee on higher education funding.
Canberra: Australian Government Publishing Service,
1988.
[33] J. S. Dawkins, Higher education a policy statement.
Canberra: Australian Government Publishing Service,
1988.
[34] T. L. J. Ferris, “A technician is half an engineer - overseas
students and advanced standing in engineering degrees,”
presented at Proceedings of the 14th annual conference
Australasian Association for Engineering Education and
9th Australasian women in engineering forum engineering
education for a sustainable future, RMIT University,
Victoria, 2003.
[35] B. S. Bloom, M. D. Engelhart, E. J. Furst, W. H. Hill, and
D. R. Krathwohl, Taxonomy of educational objectives the
classification of educational goals handbook I: cognitive
domain. London: Longman Group Ltd, 1979.
[36] D. R. Krathwohl, B. S. Bloom, and B. B. Masia, Taxonomy
of educational objectives the classification of educational
goals handbook II: affective domain. London: Longman
Group Ltd, 1973.
[37] B. S. Bloom, “Reflections on the development and use of
the taxonomy,” in Bloom's taxonomy a forty-year
retrospective, vol. 93, part 2, Yearbook of the national
society for the study of education, L. W. Anderson and L.
A. Sosniak, Eds. Chicago, Illinois: The University of
Chicago Press, 1994, pp. 1-8.
[38] T. L. J. Ferris, “Consistency of student performance
throughout Bachelor of Engineering (Electronic
Engineering),” presented at Proceedings of the 9th annual
convention and conference putting students back into
engineering education, University of Ballarat, Victoria,
1997.
[39] J. Biggs, “Teaching across and within cultures: the issue of
international students,” in Learning and teaching in higher
education: advancing international perspectives, R.
Murray-Harvey and H. C. Silins, Eds., 1997, pp. 1-22.
[40] W. E. Deming, Out of the crisis: quality, productivity, and
competitive position. Cambridge, Cambridgeshire:
Cambridge University Press, 1986.