1
This is an Accepted Manuscript of an article published by Palgrave in Higher Education
Policy, 2014, 27, pp. 323-340, first published online on 27 August 2013, available at:
http://www.palgrave-journals.com/hep/journal/vaop/ncurrent/full/hep201327a.html
Student assessment in Portugal: Academic practice and Bologna policy
Cristina Sin and Maria Manatos, CIPES (Centre for Research in Higher Education Policies), Rua
1º Dezembro 399, 4450-227 Matosinhos, Portugal
Correspondence: [email protected]
Abstract
This paper investigates institutional policies and academic practices of student assessment in
four Portuguese higher education institutions (HEIs) in the wake of European policy
developments driven by the Bologna Process. Specifically, it examines the correspondence
between European policy recommendations related to student assessment (promotion of
student-centred learning by the Bologna Process and the European quality assurance standard
and guidelines on student assessment) and actual assessment procedures in the investigated
Portuguese HEIs. It concludes that despite student-centred methodologies having started to
make inroads, it does not emerge clearly how far changes (and practices) have been driven by
recent European policy. Another notable aspect is that despite apparent institutional
compliance with national or European orientations meant to improve the student experience
of assessment (as reflected in policy documents), academic practices and students’
experiences sometimes tell a story of resistance and enduring academic beliefs and traditions.
Introduction
The Bologna Declaration was signed in 1999 by the higher education ministers of 29 European
countries, officially launching the Bologna Process whose ultimate aim was to establish the
European Higher Education Area (EHEA) by 2010. Bologna’s endeavours to generate more
synergy between European higher education systems through the pursuit of a wide range of
objectives have resulted in transformations such as reorganisation of degree structures, new
qualifications, quality assurance reforms, or increased emphasis on lifelong learning. In later
years, a concern with the substance (rather than the structure) of education has become
visible in Bologna policy, through a growing emphasis on a new pedagogic approach and the
concept of student-centred learning. Whereas the Trends III report (Reichert and Tauch, 2003)
referred only once to student-centred learning, the 2010 report discussed the concept
extensively and described the shift to student-centred learning as the ultimate measure of the
success of the Bologna reforms (Sursock and Smidt, 2010, 32). The latest Bologna
Communiqué defined as a priority for national systems, in cooperation with HEIs, to ‘establish
conditions that foster student-centred learning, innovative teaching methods and a supportive
and inspiring working and learning environment’ (Bucharest Communiqué, 2012, 5). Sursock
2
and Smidt (2010) list a set of characteristics of student-centred learning, and practices
occurring in European HEIs, which they claim to have been driven by the Bologna Process.
Some of these impinge on assessment: a shift in focus from the teacher and what is taught to
the learner and what is learnt, or on learning outcomes; concern with critical thinking and
deeper understanding rather than with knowledge transfer; formative assessment and
continuous feedback rather than summative assessment (Sursock and Smidt, 2010, 31-32).
In addition, the European Standards and Guidelines (ESG) for Quality Assurance, also
developed in the context of the Bologna Process, include one standard on student assessment
(Standard 1.3). This states that ‘students should be assessed using published criteria,
regulations and procedures which are applied consistently.’ Assessment is described as ‘one of
the most important elements of higher education’, since its outcomes ‘have a profound effect
on students’ future careers’. Among others, the guidelines recommend that student
assessment procedures shall be designed to measure the achievement of learning outcomes;
be appropriate for their objectives; have clear and published criteria for marking; not rely on
the judgements of single examiners, if possible; be accurately defined, regulated and
communicated to students (ENQA, 2005, 16-17).
However, Bologna policy travels a tortuous implementation path. Three particular features of
the Bologna Process combined make it unlikely for intentions at European level to be faithfully
translated in enacted policy, that is, in ground-floor practice in the institutions and countries
affected. First, the non-statutory character of the Bologna Process gives countries and
institutions generous leeway regarding the depth and breadth of uptake and embedding.
Second, the diversity of actors involved in its enactment – the academic community playing a
key role – renders the process ‘bottom-heavy’ (Cerych and Sabatier, 1986) and creates an
effect of diffusion of authority throughout the system. Third, Bologna policy rarely goes
beyond statements of intent, giving little indication of concrete means for the
operationalisation of the proffered intentions. The vagueness of policy texts leaves it up to
individual actors to decide on practical actions, largely determined by the characteristics of the
local context. To cite Stephen Ball, ‘the more ideologically abstract any policy is, the more
distant in conception from practice, the less likely it is to be accommodated in unmediated
form into the context of practice’ (Ball, 1994, 19). This resonates greatly with Bologna policy-
making which pursues broad general objectives meant to establish the EHEA; however,
countries choose how, when, and in which order they address the objectives depending on
national circumstances and priorities. Furthermore, when moving from national to institutional
level, a variation in responses to policy is encountered, too. How individual institutions and
departments address Bologna objectives related to student-centred learning and assessment
and the practices they develop are likely to vary from setting to setting.
In Portugal, a new student-centred pedagogic paradigm permeated the political discourse
during the implementation of the Bologna Process. A new policy concern with the outcomes of
student learning was evident, student competences receiving extensive attention in official
documents. For instance, Decree-Law 74/2006, which aligned Portuguese degrees to the
Bologna cycles, highlighted repeatedly the centrality of competences in the new study
architecture and stated that ‘a core issue in the Bologna Process is the transition from a
3
passive education paradigm based on the acquisition of knowledge to a model based on the
development of competences’ (MCTES, 2006, 6). However, how this was to manifest itself in
the pedagogical process was not spelled out. Neither has training been offered in this area.
This led to the emergence of uncoordinated, scattered practices which academics judged to be
student-centred (Sin, 2012).
Specifically on assessment, Decree-Law 42/2005, passed in the aftermath of the Bologna
Process, contains some provisions on assessment procedures in higher education. Alongside
establishing the adoption of the European credit system (ECTS), it stated some generic aspects:
e.g. scale definition, pass mark of 10, and comparability between the Portuguese quantitative
scale and the European scale (A, B, C, D, E). Beyond this, the principle of pedagogical autonomy
enshrined in Law 38/2007 granted HEIs decision-making power regarding their student
assessment procedures and methods. However, according to regulations issued by the Agency
for Assessment and Accreditation of Higher Education, legally established in 2007, the
accreditation of study programmes requires these to demonstrate the coherence between
learning outcomes and methodologies of teaching, learning and assessment. It thus echoes
Biggs’ concept of constructive alignment, according to which all the pedagogic elements –
curriculum and intended outcomes, teaching methods, assessment tasks – are in tune with
each other. Consequently, learners construct their own learning through relevant activities
driven by such a pedagogic approach (Biggs, 2003).
Against this backdrop, the paper investigates institutional and academic practices of student
assessment in four Portuguese HEIs in the wake of European policy developments driven by
the Bologna Process. Specifically, it examines the correspondence between Bologna policy
with a bearing on student assessment (promotion of student-centred learning in general and,
particularly, the European standard on student assessment) and the assessment procedures
observed in the investigated Portuguese HEIs. It aims to contribute to research, still sparse,
into ground-floor academic practices and the student experience further to recent Bologna
policy, thus enriching the still thin evidence base on institutional and academic enactment.
Approach
Theoretical considerations
The research question this paper pursues is how Bologna policy advocating student-centred
learning and assessment has trickled down to national, institutional and practitioner levels in
Portugal, impacting on student assessment. It therefore appeared meaningful to approach the
study through the lens of policy theories (Ball, 1994; Cerych and Sabatier, 1986; Ozga, 2000;
Trowler, 2002). A focus on actors’ interpretation and enactment of policy – as opposed to a
‘rational-purposive’ model (Trowler, 2002) which assumes linear, unproblematic
implementation – emerged as particularly relevant, in light of the bottom-heaviness nature of
the field of higher education referred to earlier. The analysis of institutional uptake of policy,
academic practices and student experience required the consideration of actors’ responses to
priorities defined at higher policy levels, as explained by Gornitzka, Kyvik and Stensaker (2005)
and Trowler, Saunders and Knight (2004).
4
The ‘implementation staircase’ proposed by Reynolds and Saunders (1987) was chosen as an
analytical lens because it conveys the idea of the situated nature of the different actors’
experience and enactment of policy, which, in the case of the Bologna Process could look as
below.
Bologna Process
National policies
Institutional policies
Academic practice
Student experience
Figure 1: The implementation staircase in Bologna policy enactment
The value of the implementation staircase resides in its metaphorical illustration of the
evolution of policy as it travels from setting to setting and from one set of actors to a different
one. Actors’ location, roles and interests shape reception and interpretation. The
implementation staircase makes agency and bottom-up influences explicit, by portraying all
actors involved in enactment as active agents who shape policy according to their own
interpretations, interests and resources, determined by their specific contextual
circumstances. Reynolds and Saunders’ implementation staircase therefore shows how policy
undergoes a process of adaptation and translation so that it becomes meaningful to those
people who have to deal with it in their day-to-day jobs. The following quote highlights the
variation between the planned, enacted and constructed reforms at the level of top policy-
makers, academics and, respectively, students (although the term ‘implementation gap’
would fit better in an engineering, top-down implementation model rather than in an
understanding of implementation as an actor-driven process):
Any idea that (...) policy will look the same at the bottom of the staircase as it looked at the top
would be naïve. Instead we find implementation gaps between the changes that are planned in
policy, the changes that are enacted in practice, and the changes as they are constructed in the
understandings of the students whose learning they were intended to affect. There are
differences, invariably, between planned, enacted and constructed changes. (Bamber, et al.,
2009, pp. 12-13)
The staircase metaphor, therefore, highlights the likely variation in actors’ responses and
experiences. The task of analysis becomes to uncover these differences, shaped by the unique
and situated experience of each actor group. Additionally, the staircase metaphor suggests
that policy messages travel up and down the staircase, leading to an ongoing re-creation of
policy – hence each group acts as both a policy receiver and policy agent, thus remaking policy
in the process of enactment. It is therefore suggested that the elaboration, or uptake, of
5
national or European policy is not necessarily synonymous with intended ground-floor change.
The transposition of national policy in institutional regulations is likely to differ from HEI to
HEI; likewise, the formulation of institutional policies provides no guarantee for embedding
and change in academic practice. As it is the latter that impacts directly on the student
experience, the concept of 'teaching and learning regimes' has also emerged as an appropriate
theoretical lens to understand ground-floor practice. It is defined as a ‘shorthand term for a
constellation of rules, assumptions, practices and relationships related to teaching and
learning issues’ (Trowler and Cooper, 2002, 222). Although expressed in individual behaviour
and assumptions, teaching and learning regimes are ‘primarily socially constructed and located
and so are relatively enduring.’ Departments are, according to Trowler and Cooper, the
primary location where regimes develop and are transmitted. Here, ‘academics engage
together on tasks over the long term’ and ‘in interaction both construct and enact culture’
(Trowler and Cooper, 2002, 222).
Data collection and analysis
The sample for this study comprised four Portuguese HEIs illustrative of the diversity of the
country's public higher education landscape (university/polytechnic sector, geography and
size). Out of concern with potential disciplinary differences, two programmes per institution
were chosen, as shown in Table 1, belonging to two major disciplinary areas in order to
represent the soft and hard categories outlined by Becher and Trowler (2001). For simplicity,
these are generically referred to as Engineering and Arts throughout the paper. The choice of
disciplines was limited by the necessity of their presence in all the selected HEIs. The research
does not lay claim to the representativeness of the selected institutions or programmes, and
the findings are limited to this sample.
HEI Type/location Faculty/School Study Programme
HEI
A
Small
university
(interior)
School of Sciences and Technology Civil Engineering
School of Arts
(Visual Arts and Design Department) Design
HEI
B
Big university
(litoral)
Faculty of Engineering Civil Engineering
Faculty of Fine Arts Communication Design
HEI
C
Big
polytechnic
(interior)
School of Technology and
Management Civil Engineering
School of Education Arts and Design
HEI
D
Small
polytechnic
(interior)
School of Technology
Civil Engineering
Plastic Arts – Painting and
Inter-Media
Table 1: Sample of HEIs, faculty/schools and study programmes
The data was mined from two sources. First, an analysis was undertaken of official institutional
documents (i.e. pedagogic, academic and assessment rules and regulations) publicly available
online, including both institution-wide and faculty/school-level regulations. The analysis
focused on the following topics: main policy provisions; criteria for marking, absence, class
attendance; stipulations regarding communication of assessment types, methods, criteria etc.
to students; the number of examiners (as a guarantee for impartiality). Second, a total of 21
semi-structured interviews were conducted, targeting two groups in each of the four HEIs:
first, institutional and faculty/school top representatives (Rector/President or Vice-
6
Rector/Vice-President; Dean or equivalent; and representatives of quality units at institutional
and faculty level) and, second, study programmes directors. These participants were chosen
because of their role in policy elaboration at institutional, departmental and programme level
and, especially in the case of the first group, because of their position of mediation between
external and internal demands and priorities. Additionally, 16 focus groups – two for each
study programme, one with academics and one with students – with around six participants
were conducted. The research participants among these target groups were recruited by the
study programme directors themselves, following our expression of interest in interviewing
teaching staff and students in the study programme they were leading. Interviews and focus
groups explored themes primarily around assessment practices and were mainly drawn from
the guidelines of ESG Standard 1.3. Specifically, the following items were discussed:
participants’ awareness of the existence of an institutional assessment policy and knowledge
of its provisions; if and how this information was shared with students; development of
assessment methods and their fitness-for-purpose (formative, summative etc.); the extent to
which assessment evaluates learning outcomes; assessment methods; compliance with official
rules and regulations; recent changes in assessment methods and drivers for these. Interviews
and focus group discussions were fully transcribed and subjected to content analysis. An
analytical framework integrating the above-mentioned themes (i.e. the topics covered in
interviews and focus groups) guided the codification of interview and focus group data.
Throughout the analysis of both policy texts and interview data, their resonance with
European policy recommendations on the student experience was an underlying concern.
Institutional policies, academic practice and student experience
This section is divided in two parts. The first one presents the findings on institutional policies
for student assessment, while the second one focuses on the assessment practices of
interviewed academics and the students’ experience of these.
Institutional policies on student assessment
Despite sparse national regulations, as mentioned earlier, a certain degree of policy
convergence across institutions has been noted. All four HEIs have institutional policies either
specifically on assessment or on academic procedures which include assessment, thus
reflecting the Standard 1.3 recommendation on ‘published criteria, information and
procedures’ for assessment. At the same time, some faculties/schools have their own
academic/assessment policies which bring an additional layer of detail to institution-wide
regulations, tailoring these to the disciplines in question. Sometimes departments, too, have
their own regulations, as in the case of the Department of Visual Arts and Design in HEI A,
meant to give coherence to assessment across courses.
As a general rule, policies contain broad regulations on possible assessment types and
methods, assessment procedures, examination periods, attendance requirements, marking,
student rights (access to exam papers, complaints and appeals). Identical exam periods,
regulated in detail in all institutional policies, apply in the four HEIs: the normal period (for
final exams), the appeal period (recurso) for re-examinations of students who have failed the
course or who wish to improve their marks, and the special period for the assessment of
special-regime students and students who miss a limited amount of credits to complete the
7
degree. Institutional policies unanimously acknowledge the circumstances of special-regime
students (i.e. working students, students in the army, student representatives on institutional
governing bodies, etc.) and indicate exceptions for them. This consideration appears to have
emerged further to the legislative changes which widened higher education participation to
non-traditional student populations. Decree-Law 64/2006 approved a new path to higher
education for students older than 23 not holding the standard entry requirements. This
generated a significant increase in their numbers. Especially with respect to working students –
the group most commonly referred to in policies – special conditions apply to their attendance
and assessment regimes. However, some interviewees, especially students, claimed that
academics sometimes appeared reticent to acknowledge special-regime student needs.
Policies either give clear information on attendance requirements, or ask programmes to
establish their own criteria. All the programmes considered here have clear attendance
requirements, compulsory for a proportion of between two thirds and eighty percent of
classes. A minimum required attendance functions as a pre-requisite to assessment, since
students can otherwise be denied the right to be assessed. Attendance can also influence the
final mark for a course, such as in the Arts programme in HEI A where students receive a mark
for their attendance record. As previously said, the circumstances of special-regime students
are taken into account, as they are exempt from such attendance requirements.
Concerning communication of assessment information to students, explicitly recommended by
Standard 1.3, all academic or assessment policies stipulate clearly teaching staff’s obligation to
inform students of assessment procedures, methods, criteria, marking etc. Usually regulations
require the inclusion of this information in course descriptions, which all policies ask to be
made available at the beginning of the semester or of the course. Some call explicitly for online
publication, whereas others state solely that the information must reach students at the
outset.
The number of examiners, also mentioned in Standard 1.3, could represent an impartiality
factor in assessment. In this respect, only one institutional policy (HEI A) and one faculty policy
(Engineering in HEI B) require the constitution of course assessment panels, consisting of
minimum three and, respectively, minimum two teaching staff. Both these institutions belong
to the university sector. In the other two HEIs, institutional regulations make no reference to
the number of examiners. However, regulations for Arts in HEI C stipulate that although the
course lecturer is responsible for assessment, marks must be certified by the head of
department and another member of the teaching staff. In the other instances, interviews
confirmed that responsibility for assessment lies solely with the course lecturer. When a
course is jointly taught by several academics, assessment will usually be shared, too.
Another observed tendency is the noticeable growing emphasis on continuous assessment: in
three of the four HEIs policies explicitly favour this form of assessment, described as student-
centred by European documents. Finally, an aspect observed across the board is the absence
of explicit qualitative assessment criteria corresponding to the different levels of the
Portuguese 0 to 20 marking scale (i.e. general descriptors of the quality levels of student work
corresponding to specific marks on the scale), thus going against the ESG recommendation for
8
‘clear and published criteria for marking’. The only indication given in policies is the pass mark
of 10, as stipulated in Decree-Law 42/2005.
Academic practices related to assessment
This section draws primarily, but not exclusively, on the interviews and the focus group
discussions. It presents the academics’ and students’ perceptions regarding the assessment
methodologies employed in the selected HEIs and degrees. It also addresses recent changes in
assessment procedures.
In all four institutions pedagogic autonomy grants academics the freedom to design
assessment methods as they consider appropriate. According to both Engineering and Arts
academics across the four surveyed HEIs, when they choose assessment methods (exams,
various tests or student tasks throughout the semester, projects etc.) they bear in mind course
objectives and typology (i.e. theoretical, practical, project-based etc.), thus suggesting
alignment is sought between objectives and methods, as recommended in Standard 1.3.
However, in the Engineering programmes of the HEIs from the university sector, academics felt
that their pedagogic autonomy was overridden by institutional regulations which favoured
continuous assessment over final exams – this might suggest academics’ preference for final
exams. For example, in HEI A, for a course to have final assessment only, the academic in
charge must justify this choice of assessment. In HEI B, regulations prescribe that at least one
course per semester must have continuous assessment.
Continuous assessment, whose benefits are normally associated with formative feedback, was
sometimes summative. For instance, in HEI B, Engineering academics’ understanding of
continuous assessment reflected mainly a preoccupation with inducing a regular pattern of
student learning along the semester rather than with its potentially formative function. In
practice, this often translated into summative mini-tests during the teaching period, with little
feedback provided to students. Academics considered such summative mini-tests, given their
perceived urgency, disruptive for student attendance and learning in other courses.
Continuous assessment, a practice supposed to benefit student learning, thus appeared
counterproductive, tiring and stressful because of the way it was implemented. According to
one academic’s perception:
Continuous assessment and the existence of different evaluations make sense if we are talking
about annual courses, to avoid the accumulation of subject matter. However, it does not make
sense in semi-annual courses and I believe it can be counterproductive to the learning process.
Several evaluations during a semester can impact negatively the normal functioning of the
courses and cause a lot of stress in the students.
This provides a clear example of how policy intentions can be re-interpreted in ground-floor
enactment: a different understanding of continuous assessment by academics, combined with
a preoccupation with compliance, could end up being more detrimental than beneficial for
student learning.
9
On the contrary, in the Arts programmes in all four HEIs academics appeared primarily focused
on the formative purpose of continuous assessment. It was the most widespread assessment
method, too. Academics and students alike discussed the practical nature of arts courses in
which students performed practical tasks and developed projects. The function of continuous
assessment during the course was to follow students’ performance and progress, to give
feedback and to guide further work. Then, the final assessment was summative, the evaluation
of the final piece of work. In HEI C, for example, practical tasks were the most important part
of student assessment. Academics also emphasised the importance of students’ self-
assessment and self-reflection on objectives and progress:
We want students to be aware of the difficulties of the course, the work they have to do and
the main goals they have to achieve. And since the practical component is very strong in the
programme, practical tasks are the most important part of student assessment.
Students also seemed to appreciate the value of the practical component, stating that, except
for some courses, they learnt more through practice. In one student’s words:
We prefer practical works to tests or exams, because we learn more with those works and with
the practice.
As opposed to Arts where continuous assessment appeared as the main assessment method,
in the analysed Engineering programmes continuous assessment and final assessment (as an
exam) seemed to coexist more. The degree of reliance on final exams varied as follows. In HEI
A, both students and academics spoke of a variety of assessment methods (exams, various
tests throughout the semester, projects), including the students’ possibility of sometimes
choosing among these. Continuous assessment was described as predominant. It was
perceived by academics and students alike as an added-value, since the courses were mainly
practical and formative assessment seemed most fit-for-purpose. Contrary to the frequency of
continuous assessment in HEI A, Engineering lecturers and students in HEI B reported that final
exams were the most commonly-employed assessment method with the exception of one
course per semester which had to be assessed continuously, as per regulations. However, as
discussed earlier, in this case continuous assessment did not stop being summative. A senior
academic criticised the tendency of overvaluing continuous assessment and advocated a
hybrid methodology, defending the final exam and its specific purpose of preparing students
to act under more stressful or tense circumstances. In HEI C, students emphasised two
assessment methods: tasks during the semester and the final exam. They complained about
the undervaluation of tasks, which they believed were more beneficial to learning than exams.
In HEI D, practical tasks were generally the assessment method in practical courses, while final
exams were employed in foundation courses. The close relationship and informal dialogue
between academics and students stood out as a positive characteristic in participants’
accounts, since they enabled flexibility and adjustment of assessment methodologies. As a
student declared:
We talk a lot with them [the professors] and ... if we have lot of works or tests they try to adjust
the assessment methods and try to help us.
The use of learning outcomes was another issue of consideration, given the significance
Bologna policy invests in these (Bucharest Communiqué, 2012; ENQA, 2005; Leuven
Communiqué, 2009; London Communiqué, 2007), deeming them to be central drivers of
10
student-centred teaching, learning and assessment methods. The first observation emerging
from this study, both in official institutional documents and during the interviews, was the lack
of a consistent Portuguese terminology for learning outcomes. Several terms which appeared
to refer to learning outcomes were used interchangeably: competences (competências),
learning outcomes (resultados de aprendizagem) or objectives (objectivos).
Thus the academic regulations in HEI A state that assessment aims to quantify the mastery of
student competences. The assessment regulations in HEI B require that course descriptions
should state objectives and learning outcomes. In HEI C, Engineering pedagogic regulations
require that the envisaged learning outcomes and student competences should be stated in
course descriptions, while assessment is described as a process which determines the extent
to which these have been reached. The Arts policy highlights assessment as having a central
role in the promotion of competences and the development of appropriate methodologies. HEI
D regulations state that assessment tests acquired knowledge, taking into account the
objectives defined in course descriptions. However, it is unclear whether these terms are
consistently employed to mean learning outcomes or, indeed, what they refer to.
A second observation is related to the dichotomy between official institutional policies and the
actual understanding and practice related to learning outcomes as revealed by focus groups.
As seen above, in all the analysed HEIs assessment or academic regulations make reference to
competences, learning outcomes or objectives. Institutional representatives at high and
middle levels also demonstrated understanding of learning outcomes and their pedagogic
function. However, a majority of the lay academics in focus groups – especially in the area of
Engineering – did not seem to have grasped the concept. In answering the question about the
link between assessment and learning outcomes, they led the discussion elsewhere. This
occurred despite the fact that they had drafted course descriptions where they identified
learning outcomes. Indeed, some central faculty representatives in HEI B confirmed that
despite institutional efforts to introduce a learning outcomes approach, these were still little
understood, implemented and had not yet triggered changes in pedagogic methods. One
academic’s statement reinforced this perception:
I think that sometimes we seem amateurs on student assessment. I believe that assessment
should be designed before the course begins, because the course objectives and the
competences that students are expected to achieve should be defined a priori.
In all institutions but HEI D (where the concept was totally obscure across the board), learning
outcomes appeared more accessible to Arts academics as opposed to Engineering ones. Some
illustrative examples follow. In HEI A, Arts academics defined learning outcomes as students’
understanding of a designer’s responsibility and role, teamwork, basic working tools, various
areas of knowledge etc. and stated that these were taken into account in course design. They
also believed that continuous and project-based assessment facilitated the evaluation of
intended learning outcomes, which also attempted to build bridges with the world of work.
The close teacher-student relationship allowed dedicated attention to each student’s progress
and their attainment of intended outcomes. In HEI C, too, an academic declared:
11
Students assessment methods are made to measure the learning outcomes of students. In our
courses we try to build assessment procedures (tests, individual assignments, group
assignments) to evaluate if students achieved the goals of the course.
Art students in HEIs A, B and C also appeared more familiar with the learning outcomes
defined in course descriptions as opposed to Engineering students, and also seemed to grasp
better the concept. They were also more satisfied with how assessment and teaching methods
reflected learning outcomes. According to one student in HEI C:
We know which are the goals that we have to achieve and the things that we have to learn,
because everything is written an explained in the programme specification. And we know what
we have to learn in our courses, because they explained that to us.
Engineering students, in contrast, struggled with understanding the learning outcomes for
their programme or courses. In HEI A, they claimed to receive no information on learning
outcomes. In HEI C, students were unsure about what these were. They did not understand
the reasons why they did certain tasks or answered certain questions in exams. Some students
declared to have comprehended only later the purpose of their previous years’ learning and
knowledge and intuitively understood the associated learning outcomes. In a student’s word:
Most of the time we do not understand why we are studying a subject or how it will be useful
for us in the future. Mainly in the first years, where there are a lot of things that we have to
learn, but we do not understand if we will apply the things in the future. Sometimes we
understand it later, and we realize why we studied those things.
The greater understanding and recognition of learning outcomes among Arts academics and
students appears facilitated by the fact that Arts courses involve the acquisition of practical
skills through hands-on product development and creation of artwork. Nonetheless,
Engineering is arguably a practical applied discipline, too, given its laboratorial component, but
this seems to hardly influence Engineering academics and students’ perception of learning
outcomes. It can be speculated whether this is due to the first years in Engineering being
dedicated to more theoretical foundation courses, and a maybe less practical nature of
courses than overall expected.
Changes in academic practices of assessment were a consideration of this research since they
could potentially result, at least partly, from new policy, i.e. Bologna’s promotion of student-
centred learning and the ESG. It was primarily academics’ accounts that shed light onto this
issue, since students’ time-span in higher education is limited and their perspectives are
unlikely to include pre- and post-Bologna experiences. However, we did get some insight into
their opinions. In the Engineering programmes, the most noteworthy change research
participants reported was a diversification of assessment methods and the increasing weight
of continuous assessment and ongoing student work at the expense of the final exam. This was
noted in three of the four institutions and was driven partly by regulations in HEI B and HEI A.
In HEI B, for instance, Engineering academics resented the increasingly prescriptive character
of assessment regulations (i.e. the rule of one course per semester with continuous
assessment only, or an earlier regulation which only allowed continuous assessment), which
they felt impinged on their pedagogic autonomy:
12
I am strongly against ‘pedagogic dirigisme’ and against the establishment of general rules and
impositions to such different courses and teaching methodologies. Some years ago teachers
could define the best assessment method and there were no impositions to follow a continuous
assessment.
In HEI A, academics are now required to write an end-of-course report and suggest
improvements. Additionally, if failure rates go over 25%, they must provide justification for this
and propose measures to tackle the problem. One academic declared to have increased the
number of tests further to student feedback. In HEI C, academics highlighted the increase in
assessed student assignments and tests as the main change in assessment procedures, and
classified it as pre- and post-Bologna. Students were strong defenders of these assessment
methods, believing them to be better for their learning. They disagreed with the assessment of
some courses by one exam only, as they learnt less than with assignments. They also
complained about the concentration of content in one exam. Mainly after Bologna, students
perceived this as problematic since the subject matter of previously long degrees was now
delivered in less time. This criticism suggests the superficial alignment of degrees to Bologna
principles and the compression of subject matter from previously longer degrees in the new
shorter ones, instead of a deep re-thinking of the curriculum.
Arts academics reported changes of a more varied nature. For instance, in HEI A the School
developed some general assessment rules to do away with the inconsistencies students
experienced between different courses. Students seemed satisfied with the functioning of
assessment, mainly because they felt that their views on how courses could be improved
mattered. As one student put it:
They always ask us about how we believe the course can be changed and improved; our
opinion counts.
In HEI B, academics noted a diversification of assessment methods as well as more systematic
course descriptions focused on the expected student achievement, thus claiming assessment
had become more reliable and appropriate for its purpose. In HEI C, changes were reported in
procedures related to the communication of assessment-related information to students
through new online platforms. In HEI D, academics said that the appropriateness of
assessment procedures for their purpose received more attention – hence their continuous
adjustment.
These changes suggest increased alignment with the guidelines of Standard 1.3. Some of them
(more continuous assessment, course descriptions focused on learning outcomes to inform
teaching and assessment, etc.) also imply an increased attention to assessment practices
aimed at enhancing student learning. Furthermore, academics described other initiatives
illustrative of increased engagement with reflective activities around teaching and assessment.
Thus, in two HEIs (A and B) lecturers write a report at the end of the course to evaluate the
fulfilment of its objectives and make recommendations on how it can be improved.
Initiatives which foster reflection and discussion around pedagogic issues, including
assessment, (and ways to improve these) also occur at faculty/school and programme level:
e.g. a relatively new teaching and learning lab in the Faculty of Engineering in HEI B which has
resulted in increased pedagogic awareness; an annual workshop launched recently by the
13
Engineering programme, also in HEI B, meant as an ongoing revision of the programme; or
regular meetings among the Arts academic staff in HEI A with the same reflection and
enhancement purpose. Academics felt that the above activities all resulted in ongoing
adjustments and improvements in pedagogic practices, including assessment. In HEI C, the
Bologna Process was claimed to have strengthened this continuous adjustment process.
Conclusion
This paper has aimed to investigate institutional policies and academic practices of
assessment, as well as whether these have been influenced by Bologna policy advocating
student-centred learning and assessment. A general finding is that assessment practices in the
four investigated Portuguese HEIs appear to increasingly embrace student-centred
methodologies and to some degree reflect European orientations regarding student
assessment, seeming to indicate some effects of policy on academic practice. This would be
consistent with previous research which reported that Portuguese academics perceived
Bologna as a window of opportunity to change the pedagogic paradigm towards student-
centred pedagogies (Sin, 2012; Veiga and Amaral, 2009). Nonetheless, it did not emerge clearly
from this research how far changes in practice have, indeed, been driven by recent European
policy. Only in some cases is direct reference made to Bologna. While we could infer from
some observed consistencies that Bologna might have driven these developments, especially
as regards institutional policies, the existence of a clear cause-effect relationship is hard to
uphold.
The implementation staircase metaphor (Reynolds and Saunders, 1987) used as a lens in this
study has helped illustrate the situatedness of the policy experience and the phases of policy
transformation and interpretation, which result in variable ground-floor enactment (Ball, 1994;
Gornitzka, Kogan, and Amaral, 2005; Ozga, 2000; Trowler, 2002). The unique set of
characteristics of the Bologna Process – namely its non-binding nature, the multiple actors and
levels involved in enactment, and the vagueness of its objectives together with the lack of
operationalisation cues – have favoured localised implementation and policy expressions in
such a way as to make sense to ground-floor actors and their contexts. To cite again S. Ball:
Solutions to the problems posed by policy texts will be localized and should be
expected to display ad-hocery and messiness (...) Given constraints, circumstances and
practicalities, the translation of the crude, abstract simplicity of policy texts into
interactive and sustainable practices of some sort involves productive thought,
invention and adaptation. (Ball, 1994, 18-19)
Moving along the implementation staircase, we have indentified the following situated
responses to Bologna policy, determined by actors’ location, roles and circumstances. First, as
we have noted, national policy to regulate assessment is rather vague and scarce in Portugal,
although it did adopt some Bologna principles. At institutional level, variation in
assessment/academic regulations has been observed, despite several commonalities probably
explained by the idiosyncrasies of the national higher education system. At the same time,
their observed alignment, although partial, with the guidelines of Standard 1.3 (definition of
procedures, informing students, etc.) might suggest that these do respond to European policy,
possibly through top institutional actors’ awareness of the European Standards and Guidelines
14
for Quality Assurance. Next, differences have been observed in academic practices especially
between the two disciplines (Arts and Engineering). Variation concerns mainly assessment
methods (final/continuous) and understanding of learning outcomes, two of the key areas
where changes can lead to student-centred learning. The observed variation in academic
practice suggests the strength of learning and teaching regimes (Trowler and Cooper, 2002)
and of their recurrent practices in the face of policy trends, despite the latter’s potential to act
as powerful catalysts of change in academic behaviour (Trowler, Saunders, and Bamber, 2012).
The analysis has revealed some examples of student-centred pedagogy: the very existence of
institutional student assessment or academic policies which give transparency to procedures;
communication of relevant information to students early on, creating additional transparency;
the inclusive character of assessment procedures (at least officially), as institutional
regulations acknowledge the needs of special-regime students by stipulating exceptions from
general attendance and assessment requirements; a growing presence of continuous
assessment alongside final assessment; increasing reflection around pedagogic practices.
However, despite apparent institutional alignment with European orientations (as reflected in
rules and regulations), and despite academics’ apparent compliance with institutional
regulations as inferred from the interviews, academic practices and students’ experiences
occasionally tell a different story. Changes as the ones highlighted above sometimes appear to
be pro-forma: the inclusive character of policies is not always reflected in practice, as
academics sometimes appear unwilling to acknowledge the needs of working students;
continuous assessment can be summative, therefore not necessarily fulfilling a formative
function meant to help students along their learning journey; learning outcomes, despite their
presence in one form or another in institutional documents, are variably understood by
academics and students and especially problematic in Engineering, hence their dissociation
from assessment; and students sometimes complain that assessment procedures lack
transparency.
This research has, therefore, revealed some areas of student assessment which could
potentially benefit the student experience if they were to receive further consideration at the
various policy levels. Thus, the poor understanding of learning outcomes might be addressed
at institutional level, but also national, through initiatives to raise awareness of the potential of
learning outcomes as a pedagogic tool in teaching and assessment design. Pedagogic training
could also tackle some of the shortcomings identified in assessment procedures, such as
academics’ reticence to acknowledge the regime of working students or the preference for
summative assessment in the Engineering degrees. At the same time, institutional regulations
could address factors which affect the transparency and objectivity of assessment, such as the
absence of clear tiered criteria to inform marking and the reliance on single examiners.
Acknowledgements
This research was undertaken in the context of the project IBAR funded by the European
Commission, entitled 'Identifying barriers in promoting the European Standards and Guidelines
for Quality Assurance at institutional level', reference 511491-LLP-1-2010-1-CZ-KA1-KA1SCR.
15
References
Ball, S. J. (1994) Education reform : a critical and post structural approach, Buckingham: Open
University Press.
Bamber, V., Trowler, P., Saunders, M., & Knight, P. (2009). Enhancing learning, teaching,
assessment and curriculum in higher education: theory, cases, practices. Maidenhead:
Society for Research into Higher Education & Open University Press.
Becher, T. and Trowler, P. (2001) Academic tribes and territories: intellectual enquiry and the
culture of disciplines, Buckingham: Open University Press.
Biggs, J. B. (2003) Teaching for quality learning at university : what the student does,
Buckingham: SRHE & Open University Press.
Bucharest Communiqué (2012) ‘Making the Most of Our Potential: Consolidating the European
Higher Education Area’, Bucharest.
Cerych, L. and Sabatier, P. A. (1986) Great expectations and mixed performance : the
implementation of higher education reforms in Europe, Stoke on Trent: Trentham
Books.
ENQA (2005) Standards and Guidelines for Quality Assurance in the European Higher Education
Area, Helsinki: European Association for Quality Assurance in Higher Education.
Gornitzka, Å., Kogan, M. and Amaral, A. (2005) Reform and Change in Higher Education.
Analysing Policy Implementation, Dordrecht: Springer.
Gornitzka, Å., Kyvik, S. and Stensaker, B. (2005) ‘Implementation Analysis in Higher Education’,
in Å. Gornitzka, M. Kogan and A. Amaral (eds). Reform and Change in Higher
Education. Analysing Policy Implementation, Dordrecht: Springer, pp. 35-57.
Leuven Communiqué (2009) ‘The Bologna Process 2020 -The European Higher Education Area
in the new decade. Communiqué of the Conference of European Ministers Responsible
for Higher Education’, Leuven.
London Communiqué (2007) ‘Towards the European Higher Education Area: responding to
challenges in a globalised world’, London.
MCTES (2006) ‘Decree-Law No. 74/2006, of 24 March’, Lisboa: Ministério da Ciência,
Tecnologia e Ensino Superior.
Ozga, J. (2000) Policy research in educational settings: contested terrain, Buckingham: Open
University Press.
Reichert, S. andTauch, C. (2003) Trends 2003: Progress towards the European Higher Education
Area. Bologna four years after: steps toward sustainable reform of higher education in
Europe, Geneva/Brussels: European University Association.
Reynolds, J. and Saunders, M. (1987) ‘Teacher Responses to Curriculum Policy: Beyond the
‘Delivery’ Metaphor’, in J. Calderhead (ed). Exploring Teachers’ Thinking, London:
Cassell Educational Limited, pp. 195-214.
Sin, C. (2012) ‘Academic understandings and responses to Bologna: a three-country
perspective’, European Journal of Education 47(3): 392-404.
Sursock, A. and Smidt, H. (2010) Trends 2010: A decade of change in European Higher
Education, Brussels: European Universities Association.
Trowler, P. (2002) Higher education policy and institutional change : intentions and outcomes
in turbulent environments, Buckingham: Society for Research into Higher Education &
Open University Press.
16
Trowler, P. and Cooper, A. (2002) ‘Teaching and Learning Regimes: Implicit theories and
recurrent practices in the enhancement of teaching and learning through educational
development programmes’, Higher Education Research & Development 21: 221-240.
Trowler, P., Saunders, M. and Bamber, V. (2012) Tribes and Territories in the 21st Century:
Rethinking the significance of disciplines in higher education, London: Routledge.
Trowler, P., Saunders, M. and Knight, P. (2004) ‘Change thinking, change practices. A guide to
change for heads of department, subject centres and others who work middle-out’,
Paper written with support from: The LTSN Generic Centre, The UK Evaluation of the
LTSN and The HEFCE Innovations project ''Skills plus''.
Veiga, A., & Amaral, A. (2009). ‘Survey on the implementation of the Bologna process in
Portugal’, Higher Education 57(1): 57-69.