Post on 10-May-2023
The Determinants of the Students’ Academic Performance in Accountancy
Courses: Evidence from Brazil
1. Introduction1
The standardized large-scale tests, so-called Achievement Tests, were instituted
by the Brazilian government on the second half of the 90-ies. The first test was the
National Exam of Courses (ENC) so-called Provão that was accomplished over the
period from 1996 to 2003 and was subsequently replaced by the National Exam of the
Students’ Performance (Enade). According to the Instituto Nacional de Estudos e
Pesquisas Educacionais AnísioTeixeira (INEP), the agency responsible for elaboration
of the tests and promotion of the studies, research and evaluations on Brazilian
Educational System, those tests were instituted as part of the evaluative policy of the
courses offered by the Higher Education Institutions (IES). The main objective of both
tests is to verify if the knowledge, academic skills and professional competencies of
each area under evaluation were developed by students throughout the course. Those
knowledge, skills and competences are previously defined by national educational
guidelines. More than 470,000 students graduated in 6,5 thousand courses of 26 areas
participated in the last edition of the ENC-Provão conducted in 2003 (INEP, 2004).
This study focuses on the Brazilian Accountancy students, who participated in
either Provão (2002 and 2003) and Enade (2006 and 2009). According to INEP, 22,694
graduating students of the Accountancy course participated in 22,694 exams in 2002,
22,976 in 2003 and 19,040 in 2006. In the Higher Education Census (CES) relative to
2009, the number of students who completed the course was 34,557. The main objective
is to analyze the effect from individual and institutional characteristics on academic
performance of the undergraduate students in Accountancy, by using the results of the
tests accomplished in 2002, 2003 and 2006. The research question guiding this study is:
how the students’ characteristics, the background your family, and Higher Education
Institutions affect the accounting students’ academic performance?
1 This study was financially supported by CAPES.
2
To answer this question, it is intended to verify the following: (1) the personal
aspects such as color, gender, age of the students are related to their academic
performance; (2) the academic performance is positively related to socioeconomic
status; (3) the characteristics of the course, such as participation in extracurricular
activities, the type of the academic material used and other inputs are related to
students’ academic performance; (4) the characteristics of the faculty in the educational
institutions, such as higher proportion of teachers with master and doctoral qualification
and the work scheme type among other aspects have a positive effect on students’
performance; and (5) how the educational institution is organized as an university or is
linked to public or private system has significant effect on students' academic
performance.
The relevance of this paper is related to low average result of the Accountancy
students at those tests, as compared to other areas evaluated in 2002, 2003 and 2006.
According to INEP (2003, 2004), the Provão editions 2002 and 2003 consisted of 40
multiple-choice objective questions with five response options, as corresponding to 60%
the exam total value, and three discursive questions corresponding to remaining 40%.
The general average of the Accountancy students was 32.0 on scale from 0 to 100, in
the Provão editions 2002 and 2003. The Enade/2006 is divided into two components.
The first component is General Training (FG) comprising eight multiple-choice
questions and two discursive questions, as corresponding to 25% the total exam. The
second part so-called Specific Component (CE), which includes 26 multiple-choice
questions and four discursive questions, corresponds to 75% the total exam value. This
exam was applied to students who were entering or graduating2 in the area. The general
average, that includes both freshmen and graduating students, was 44.1 in exam of the
General Training (FG) and 25.7 in exam of the Specific Knowledge (CE) (INEP,
2007a). In this last component, the average of the graduating students was 30.0. Among
all 15 areas3 under evaluation, the performance of the Accountancy area in FG was only
ahead Administration (42.1). Moreover, the students obtained the lowest general
average in CE, among all 15 areas3 under evaluation in 2006. This performance of the
2 Freshmen are students who attended the first year, whereas the graduating are those students who are at
the end of last year of the course. These groups of students were subjected to the same exam in
ENADE/2006
3 Administration, Archivology, Library Science, Biomedicine, Accountancy, Economics, Media, Design,
Law, Teacher Education, Music, Psychology, Executive Secretary, Tourism and Theatre.
3
area is among the worst ones, when compared to 15 areas evaluated in 2006 (INEP,
2007). In 2009, the general average achieved by graduating students in Enade was 39.9
in FG exam and 32.6 in CE exam (INEP, 2011).
Em 2009, a média geral obtida pelos estudantes concluintes da área no Enade foi
39,9, na prova de FG, e 32,6, no teste de CE (INEP, 2011). Em 2012, a média geral
obtida pelos estudantes concluintes da área no Enade foi 39,4, na prova de FG, e 32,8,
no teste de CE (INEP, 2011). Na prova de conhecimento específico da área a média dos
concluintes de 2009 foi inferior a 50,0.
However, in either Provão 2002/2003 and Enade/2006, the students indicated
difficulties to answer the exam questions4. The main options noted in Provão/2003
were: they studied the most test contents, but have already forgotten them (34.4%) since
there was a long time they studied most of them; and they studied many of those
contents during the course, but with little understanding (44.2%) (INEP, 2004). In
Enade/2006, a change occurred in the response pattern: only 4.5% graduating students
pointed out non-acquaintance of the content as factor explaining their performance. The
most frequently mentioned reasons were the different way to approach the content in
relation to what is usually used (41.1%) and lack of motivation (35.4%) (INEP, 2007a).
In Provão/2003, the percentages relative to those aspects were 50.6% and 18.8%,
respectively (INEP, 2004). These results of the students' perceptions are evidence for
possible problems in the teaching process, which may have negatively affected the
results of the evaluations.
The accountancy course has strong credencialism characteristics. The
professionals have their privileges guaranteed by legal act. For this reason, the large-
scale evaluations are so important. According to data from the Conselho Federal de
Contabilidade (CFC) relative to 2010a, (the brazilian agency that resembles the
American Institute of Certified Public Accountants - AICPA), there were 292,390 active
accountant records and 76,283 active organizations registered, as being 48,731 under
individual scheme and 27,552 as companies. It is important to emphasize the tendency
for more rigorous requirements concerning to professional accounting practice in Brazil.
The next students graduating in Accountancy will face other mechanisms that will
4 At the end of the test, there is a section in which students express their opinion on this evaluation tool.
4
restrict their entry and permanence in the labor market as Accountants, such as the
Sufficiency Exam and continuing education programs established by Law n. 12249
(2010). According to CFC, in the first edition of the Sufficiency Exam accomplished in
2011, only 30.83% participants who required the Accountant registration were
approved, a percentage considered low by CFC committee members responsible for
accomplishment of the Exam (CFC, 2010). This exam is similar to that accomplished by
AICPA for obtainment of the Certified Public Accountants (CPA).
This scenario provides evidence that professional training and qualification can
have preoccupying deficiencies. In Brazil, as pointed out by Crespo and Reis (2009), a
reduction in importance of the employee to have only a bachelor’s degree (in terms of
the remuneration differential) occurred over the period from 1982 to 2004, due to the
increased supply of qualified professionals. This fact, coupled with the increased
requirements for professional practice, emphasizes the importance of the accounting
education quality.
The main scientific works that motivated the use of the theoretical framework of
the economics of education were Hanushek (1979, 1987) and Hanushek and
Woessmann (2011). In Brazil, the studies conducted by Albernaz, Ferreira and Franco
(2002), Machado et al. (2008), Franco and Menezes-Filho (2009) particularly in
economics area have emphasized the primary education. It is possible to identify
internationally and in the case of Brazil, a pattern of results that points out a positive
and significant effect of the socioeconomic background on students` performance at all
educational levels.
There is significant international literature on education at higher level. For
example, Betts and Morell (1999), using the data from 5,000 undergraduate students at
the University of California, concluded that the personal background, the origin of high
school and the experience level of the high-school teachers significantly affect the
students’ performance in Grade Point Average (GPA). Cohn, Cohn, Balch and Bradley
Jr. (2004) evaluated the degree at which the score in the Scholastic Aptitude Test
(SAT), the average score in GPA and a categorical classification of the secondary
schools predict student performance on GPA, based on data from University of South
Carolina relative to period from 2000 to 2001. The results suggest the SAT score to be
related to academic success at graduation level. However, its requirement can reduce the
5
chances of the men and non-white students to be selected for a scholarship. Horowitz
and Spector (2005) used the GPA in order to verify the impact of the secondary school
(public x private) upon performance of 15,270 students enrolled at Ball State University
and proceeding from secondary school (public x private), during a semester in 2002.
The authors concluded that, although the effect is small, the students from religious
secondary schools achieved better performance than the students from private and
public schools. However, this effect disappears during graduation.
In Brazil, it is highlighted the scientific works carried out by Soares, Ribeiro
and Castro (2001), Gracioso (2006), Diaz (2007), Moreira (2010) and Rezende (2010),
who focused on context of other knowledge areas. Soares, Ribeiro and Castro (2001)
compared the score obtained by students in the selection test (vestibular) for higher
education at the Federal University of Minas Gerais State, with Provões of Law,
Administration and Civil Engineering courses, which were accomplished over the
period from 1996 to 1999. The evidences suggested a significant influence from the
socioeconomic conditions and previous performance of the students on their academic
performance. Gracioso (2006) focused upon effect of the school and concluded that the
use of microcomputers by students, the skills developed throughout the course and
English skills are important to explain the performance of the Administration Course’
students in Provão 2003. Moreira (2010) investigated the effect from the institutional
factors on students’ performance in areas of Biology, Civil Engineering, History and
Pedagogy in Enade 2005. The author found significant variations among the effects
from the institutional factors upon performance, as depending on the course and
administrative IES category. Diaz (2007) focused on performance of the higher
education students in the Management, Law and Civil Engineering courses, who
participated in Provão 2000. The family income has significant influence, but in a
nonlinear way. The institutional factors such as qualification and working conditions of
the faculty and the use of research activities as strategy for teaching/learning were
significant, but with reduced magnitude. Rezende (2010) analyzed the effect from
policy adoption of an accountability system in Higher Education, particularly during the
ENC period, from 1996 to 2003, upon IES performance. The evidences of this study
indicated that ENC policy had positive effect on the proportion of the teachers’
exclusive dedication, number of the vacancies offered, candidates inscribed, and
6
students registered. According to the author, this effect is differentiated according to
IES organizational structure.
In Accountancy context, the studies carried out by Andrade and Corrar (2007),
Cruz, Corrar and Slomski (2008) and Souza (2008) are distinguished. Andrade and
Corrar (2007) also used the data from Provão 2002 and, by mean tests, he concluded
that the academic (research activities, dedication to studies) and economic variables
(income, parental education level) are related to different academic performance of the
students, except for different races (majority x minorities) and frequency in using the
library. Cruz, Corrar and Slomski (2008) used the mean tests and concluded the updated
domain of the discipline contents, the teaching technique employed and the type of the
didactic resource used in classroom to be significantly associated with differences in the
educational performance achieved by students in Provão 2002. Souza (2008) used the
Course Performance Index (IDC) of Enade 2006 as proxy for educational outcome.
According to results, the following variables significantly affected the students´
academic performance: the high school from which students came, the father's
educational level, the student’s personal effort in the course, the fact the student to work
or not and family income. Another finding was the negative correlation between
maternal education and performance in the course. However, on the whole, it was
observed that parental education has positive influence on prediction of the students’
performance in the accountancy courses, in Enade. However, those studies were not
based on education economy field and their analyses were based on mean tests and
cross-section regressions.
This study aims to provide an original contribution, when basing the analyses on
more recent nationwide database and focusing on a test used as political evaluation of
the academic knowledge obtained during the undergraduate course and not on a test for
Admission of students to graduation. Furthermore, this article focus on student’s
performance in Accountancy in Brazil during the years 2002, 2003 and 2006, by using
the theoretical framework of the economics of educations. In general, our results
suggest the existence of significant association between the graduating students’
academic performance with certain characteristics, such as hours dedicated to studies, to
have attended the public high school, and some inputs from educational institutions
such as teachers who master properly the content of the courses. Those results can be
7
helpful to analyze the implications of either individual choices and educational policies
adopted by institutions concerning to students’ performance.
This article has four sections, besides this introduction. The next section
describes the Brazilian Higher Education System and the academic achievement tests.
The third section presents the hierarchical linear models of the academic performance.
The fourth section describes the data set and analyzes the results. The fifth section
presents the final comments.
2. The Higher Education System and Tests of the Brazilian Academic Performance
2.1. The Brazilian System of Higher Education
In Brazil, the courses offered by the Higher Education System (SES) are
distributed between the public and private IES. These undergraduate courses are offered
under the following modalities: presential, on-line courses (EAD) and regular courses.
The SES places are offered by different organizational structures, by
administrative category and academic organization. The classification based on
administrative categories encompasses the federal, state, municipal and private
institutions, whereas the categorization by academic organization comprises the
universities, university centers, integrated colleges, colleges, superior schools and
institutes and technical education centers. The knowledge areas offered by SES are
heterogeneous and generally refer to traditional careers in the labor market, such as
Medicine, Nursing, Administration, Civil Engineering, etc.
The Brazilian government has continuously adopted policies to increase the
supply of places and expansion of SES. According to CES consolidated data, this policy
was evident from the 70s. The actions of those policies are reflected on evolution of the
number of places offered for Superior Education (ES); in 1970, they totalized 145,000,
whereas in 1971 they reached 202,110, as increasing 39.4% and ending the decade with
402,694 places offered in 1979. Despite the slower rhythm, this growth trend continued
in the 80s, with 15.3% increase in places offered, that is 404,814 in 1980 to 466,794 in
1989. In the 90s, the increase was 92.8%. The outcomes from those policies can also be
observed in the enrollment growth rate between 1995 and 2009 in SES courses.
According to Fig. 1, it increased over the period from 1997 to 2002, as reaching a peak
of 14.8%. From 2003 to 2009, however, there was a downward trend and, in last year, a
8
decrease of 1.2% occurred. In the same period, the line representing the Accountancy
Course in Fig. 1 follows this downward trend, expect for growth peaks of 8.1 and 24.1%
in 2005 and 2007, respectively. These peaks are reflections especially from increases in
number of the students enrolled in EAD courses, according to data of the Higher
Education Census (CES). It is noted the registration rate of the Accountancy Course in
EAD modality to vary greatly. According to CES data, there was 77.4% reduction in
enrollments from 2005 to 2006, succeeded by an increased rate of 1,188.7% during the
period from 2006 to 2007 and an increase of 174.8% over the period from 2007 to 2008
and a decrease of 10.4% from 2008 to 2009.
SOURCE: Microdata of the Higher Education Census from 1995 to 2009.
Fig. 1. Variation rate of the enrollments in undergraduate courses in Brazil and in Accountancy courses
between the years from 1995 to 2009.
In Brazil, although there is public funding programs for higher education5, it is
noted the most spending on ES to come from private sources, since according to report
by CES 2009, 74.4% from the total 5,954,021 students enrolled in ES are affiliated to
private institutions (INEP, 2010). Among those students from private IES in 2009, only
27.9% obtained the scholarship, as being 1,019,532 from reimbursable programs and
215,777 from non-reimbursable ones (INEP, 2010). In 2009, 72.1% students in private
5 For example, the Student Financing Program (FIES) and University for All (ProUni).
6,2%
4,1%
9,3%
11,5%
13,7%12,5%
14,8%
11,7%
7,1%
7,0%5,0%
4,4% 4,1%
0,7%5,2% 3,2%
5,3%4,3%
2,2%
5,0%
7,7%7,1%
2,6%
8,1%
3,2%
24,1%
4,5%
0,3%0%
5%
10%
15%
20%
25%
30%
1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
Total Brazil Total Accountancy
9
IES paid their studies with their own resources, with their family support or through
other sources.
According to CES 2009, the Brazilian SES offered 28,671 undergraduate
courses, from which 20,043 were linked to private IES and 8,628 to public institutions,
where the average number of students registered by course amounted 221 and 177,
respectively. The distribution of the students enrolled by IES were, on average, 2,141
students per private institution, whereas in public IES were 6,220. In case of the
Accountancy area, object of this study, Table 1 shows that historically, from 1995 to
2009, the number of students enrolled in private IES were higher. However, the average
number of the students enrolled in public IES were higher than in the particular ones.
The ratio between enrollment numbers and course numbers may indicate the
concentration of students enrolled at lower number in the public IES, in contrast to
lower enrollment distribution in higher number of private IES.
Table 1
Evolution of the number of Accountancy undergraduate courses by Administrative Category from 1995 to
2009
Number of courses Number of the enrolled students Enrollments / Courses
Year Public Private Total Public Private Total Public Private Total
1995 105 247 352 33,389 73,749 107,138 318 299 304
1996 111 273 384 35,536 77,215 112,751 320 283 294
1997 112 275 387 36,368 80,023 116,391 325 291 301
1998 114 293 407 37,040 85,480 122,520 325 292 301
1999 100 359 459 34,154 93,606 127,760 342 261 278
2000 116 394 510 33,011 97,502 130,513 285 247 256
2001 117 461 578 33,659 103,330 136,989 288 224 237
2002 125 516 641 35,133 112,342 147,475 281 218 230
2003 136 565 701 37,046 120,945 157,991 272 214 225
2004 139 624 763 37,929 124,221 162,150 273 199 213
2005 139 679 818 42,218 132,987 175,205 304 196 214
2006 143 754 897 37,440 143,352 180,792 262 190 202
2007 146 794 940 58,455 165,940 224,395 400 209 239
2008 150 857 1.007 47,044 187,557 234,601 314 219 233
2009 153 899 1.052 47,668 187,606 235,274 312 209 224
SOURCE: Microdata of the Higher Education Census from 1995 to 2009. Tabulations made by the
authors.
According to CES 2009, the colleges represented 85% the total IES and the
isolated educational units so-called colleges, schools, institutes, integrated colleges and
technology colleges were considered as isolated units. In 2009, the representativeness of
10
the universities and university centers were 8% and 5.5%, respectively. The IES
distribution by administrative category was 89.4% private institutions and 10.6% public
institutions fractionated on 4.1% federal, 3.6% state and 2.9% municipal institutions.
According to Fig. 2, the number of the presential Accountancy courses, by academic
organization, increased in the University Centers. It is observed the occurrence of
concentration in the organizational structures of colleges and universities.
0
200
400
600
800
1.000
1.200
1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
15
3
15
9
18
8
19
3
20
4
23
2
24
1
24
3
25
9
27
7
29
0
31
0
32
9
34
3
33
3 33
43 50 58 64 86 93 1
00
97 10
9
12
1
59 69 50 53 39
44 48 50 58
56 55 51 59
14
0
15
6
14
9
16
1
18
2
19
0
23
8
28
8
31
6
33
9
37
4
42
2
43
8
53
3
57
4
Universities University Centers Integrated Colleges
Colleges, Schools, Institutes Centers of Technological Education
SOURCE: Microdata of the Higher Education Census from 1995 to 2009
Fig. 2. Evolution of the number of the Accountancy undergraduate courses by
academic organization under classroom modality from 1995 to 2009
The presential courses accounted for 86% from 5,954,021 students enrolled in
Brazil (INEP, 2010). In 2009, a total of 838,125 students were enrolled in EAD courses
and 665,429 from those enrollments were accomplished in private IES and 172,696 in
public IES (INEP, 2010). The indicator of the enrolled student number pointed out that
each presential course attended, on average, 184 students enrolled, whereas the average
number of the students enrolled in each EAD course were 993 in 2009. Those values
show the achievement level of the EAD courses, because the number of the attended
students by course surpassed the presential modality, although the number of the
courses were reduced. Table 2 presents the increase of the courses under EAD modality
in Brazil: they were 10 in 2000 and totalized 844 in 2009. This increase tendency is
accompanied by Accountancy area.
11
Table 2
Evolution of the number of undergraduate courses in Brazil and the Accountancy Course in classroom
and without the presence in classroom over the period from 1995 to 2009.
Total Accountancy Course Representativeness
Year Presential EAD Total Presential EAD Total %
Presential
%
EAD
%
Total
1995 6,252 0 6,252 352 0 352 6 0 6
1996 6,644 0 6,644 384 0 384 6 0 6
1997 6,132 0 6,132 387 0 387 6 0 6
1998 6,950 0 6,950 407 0 407 6 0 6
1999 8,878 0 8,878 459 0 459 5 0 5
2000 10,585 10 10,595 510 0 510 5 0 5
2001 12,155 14 12,169 578 0 578 5 0 5
2002 14,399 46 14,445 641 0 641 4 0 4
2003 16,453 52 16,505 701 0 701 4 0 4
2004 18,644 107 18,751 763 0 763 4 0 4
2005 20,407 189 20,596 816 2 818 4 1 4
2006 22,101 349 22,450 886 11 897 4 3 4
2007 23,488 408 23,896 923 17 940 4 4 4
2008 24,719 647 25,366 985 22 1,007 4 3 4
2009 27,827 844 28,671 1,028 24 1,052 4 3 4
SOURCE: Census microdata of the Higher Education from 1995 to 2009. Own tabs.
Along the period under study, most IES are private; there is lower concentration
of the students enrolled in public IES, who are also at lower number; and the enrollment
rate tends to decline. This reduction may be due to existence of idleness, since
occupancy rates show that over 50% places are not filled in ES. This context can
generate the SES closing and merging of institutions, possible adjustment of costs and
fees charged by private IES for their services and the loosening of the selection process
of future students.
It is emphasized that the scenario presented above refers to the period after
implementation of the SES regulatory policies, that were institutionalized especially in
the 90s. Those policies helped the implementation of either measurement tools and
evaluation of the players involved in the SES. Those tools were developed in order to
create quality of education indicators to subsidize the systematization of the evaluation
process of the quality of Higher Education and the national policies.
12
2.2. The Brazilian Academic Performance Tests
In order to evaluate the skills and academic performance in SES ambit, some
tests were institutionalized by the Brazilian government through Law no. 9131 (1995),
which created the National Course Exam (ENC). This exam so-called Provão was
conducted during the period from 1996 to 2003 and can be considered as starting point
for implementation of an evaluation system of the higher education in the country.
Besides ENC, later the Higher Education Census (CES) and the Evaluation of the
Education Conditions (ACE) were created, through visits of the external committees to
Higher Education Institutions (IES). INEP, a federal agency affiliated to Ministry of
Education and Sports, the current Ministry of Education (MEC), has the responsibility
for accomplishing periodic evaluations in institutions and higher level courses. This
process is intended to evaluate the quality and efficiency of teaching, research and
extension.
In 1996, INEP started the implementation of the Provão, by applying the first
exam for courses in Management, Law, and Civil Engineering. Until the end of the
exam in 2003, twenty six areas of knowledge participated in the test (INEP, 2004, p. 5).
The organization, elaboration and application of the exams were in charge of INEP in
partnership with other institutions. The ENC-Provão was annual and should include
minimum contents established and reported previously, for each course. The inclusion
of minimum contents aimed to assess the knowledge, skills and abilities acquired by
students at completion stage of the undergraduate courses (INEP, 2002, 2003). The
exams could be composed by multiple choice questions and discursive ones or exams
entirely consisting of discursive questions (INEP, 2003a). The adopted exam model was
under responsibility of each Course Commission (INEP, 2003a).
The Law n. 9131 (1995) defined the results from evaluation of the courses to be
broadly divulged to society by MEC. Those results were recorded on the student's report
description, concepts from A to E. The results from Exam and the reports by experts
affiliated to MEC were the basis for the Board of Higher Education (CES) to decide
about IES periodic recertification, according to Law n. 9131 (1995). This determination
allowed for decertification of courses that, according to CES, were not meeting the
educational quality standards previously established by MEC. For example, those
courses that obtained the concepts D and E in the last three exams could be closed.
13
In addition to Exam, each enrolled student received in advance, at his/her
residence, a questionnaire-survey, which should be devolved on exam day, from which
the goal was to collect the sociocultural and socioeconomic data and the student´
expectations regarding the characteristics of the courses, the resources and available
facilities, curriculum structure and teacher performance (INEP, 2003a). Students also
completed a questionnaire of impressions soon after exam, in order to generate
information that aimed at improving the next exams, particularly in relation to their "[...]
clarity and objectivity of statements, adequacy of information provided for resolution of
the questions, adequacy of the time for accomplishment of the exam and the difficulty
level and extent of the exam "(INEP, 2003a, p. 44).
In 2004, through Law no. 10861 (2004) which established the National System
of Higher Education Evaluation (SINAES), the Brazilian government made changes in
ES evaluation process and replaced the ENC by the National Exam of Student
Performance (Enade). Both exams aim to get knowledge, academic skills and
professional competencies, previously defined by educational guidelines developed by
the students along the course. Those aspects were evaluated at different stages of the
student’academic life. In the case of Provão, the exam was applied to graduating
students.
There are two important differences in Enade. The first one is that two distinct
groups of students, at different graduation times, are selected at random to participate in
exam. The first group, considered as entering students, must be coursed until the end of
the first year. The second group, the graduating students, are at the end of the last year
of the course. Both groups answer the same questions. The INEP purpose is to evaluate
the students’ performance throughout the course, with regard to knowledge, academic
abilities and professional skills.
The second difference is that Enade is divided into two components: General
Education (FG) and the Specific Component (EC). The questions of the General
Education component, common to all courses, seek to evaluate "the formation of an
ethical and competent professional who is committed to society in which he lives"
(INEP, 2007). The questions of the Specific Knowledge component seek to evaluate
whether students have developed, in the training process, the knowledge, skills and
14
competencies predefined by specialist committees, which are based on the National
Curriculum Guidelines for each course under evaluation.
For elaboration of the exam and definition of the desired objectives in relation to
attributes cited, INEP relies on specialist committees composed by teachers with
minimum Master title and yearly appointed by INEP. As FG exam is unique, only one
committee is responsible for its elaboration.
Thus, based on definition by Ryans and Frederiksen (1955), it is observed the
tests such as Provão and Enade to function as tools for measuring the educational
standards from which the objects or phenomena to be measured are the knowledge,
skills and individual achievements. In this study, the term ‘academic achievement’ will
be characterized according to Helmke and Schrader (2001), as the result from cognitive
learning produced by the process of education or cognitive knowledge that is intended
to teach in the school context. According to the authors, those cognitive results mainly
comprise the declarative knowledge of how to make and the individual ability to solve
problems and develop strategies.
2.2.1. The Academic Performance Tests in the Accountancy Course in Brazil
The Provão, that lasted four hours, in 2002 and 2003 consisted of 40 objective
multiple-choice questions, with five response options, as corresponding to 60% the total
value of the exam, and three discursive questions corresponding to remaining 40%. The
Enade 2006 and 2009 also consisted of 40 questions. The weights for multiple-choice
questions and discursive ones and the division of contents into FG and EC were equal in
both years. However, the number of CE questions in 2006 was changed in 2009, as
shown in Table 3.
Table 3
Summary of the Accountancy course questions in Enade 2006 and 2009
2006 2009 2006/2009 2006/2009
Components/Question types Number of
questions
Number
of
questions
Weight
(%)
Weight of the
component (%)
General Formation/Multiple choice 08 08 60 25
General Formation/ Discursive 02 02 40
Specific Component/Multiple choice 26 27 85 75
Specific Component/Discursive 04 03 15
SOURCE: Adapted from Enade Exams 2006 and 2009
15
Based on Ministerial Ordinances n. 2250 (2001) and n. 3187 (2002) and INEP n.
80 (2006) and n. 83 (2009), it may be noted that all committees responsible for
elaborating the guidelines for knowledge question contents in the Accountancy area
were composed by teachers with at least Master degree, from different Brazilian regions
and different types of institutions. In addition, an INEP-conducted study analyzed the
educational projects of the courses under evaluation for the commission to establish
guidelines for elaboration of the exam (INEP, 2004).
Furthermore, by means of guidelines of the Provão 2002/2003 established by
Ministerial Decrees n. 3018 (2001a) and n. 3818 (2002a), it was verified that no great
changes occurred in their content over the period from 2002 to 2003. It is only noted a
more detailed specification of the desired profile, skills and abilities to be evaluated. In
the guidelines established by INEP Ordinances n. 121 (2006a) and n. 125 (2009) in
Enade years, the increase referring to component of the FG exam occurred. The main
changes in formats of both Provão and Enade were decided by committees, especially in
relation to specific knowledge.
Those guidelines were the basis for guiding the elaborators of the exams on how
to evaluate the contents. For example, in all tests, there was no specific question to
evaluate the student's competence to adequately express in Portuguese (language spoken
in Brazil). In the discursive questions, however, this competence was required. The
ethical behavior and the contents of Administration, Economics and Law were broached
in questions with focus on Accountancy, but with interdisciplinary nature. The
knowledge of mathematics and statistics were required to solve questions of other
topics. So, besides seeking to verify the content knowledge, each question was designed
to capture a set of interdisciplinary skills. For example, in Provão 2002, in the same
question were collected knowledge in law, professional ethics and also logical
reasoning, analysis and emission of critical judgments.
3. The Hierarchical Linear Model of Academic Performance
An econometric technique often used to estimate the educational production
function is the regression by Ordinary Least Squares (OLS) (Betts & Morell, 1999;
Cohn, Cohn, Balch, & Bradley JR., 2004; Horowitz & Spector, 2005). However, some
authors such as Hanushek and Woessmann (2011) show that although the robustness of
the method, its estimates need to be cautiously interpreted when a broad range of inputs
16
is used because it can generate estimation biases in the coefficients of variables because
endogeneity. Due to this fact, some studies have used the regression techniques with
hierarchical structure (Soares, Ribeiro, & Castro, 2001; Albernaz, Ferreira, & Franco,
2002; Diaz, 2007). According to Raudenbush and Bryk (2002), the Hierarchical Linear
Models (HLM) enable the analysis of data which structure indicates the existent of
correlation among individuals pertaining to the same group.
There are different HLM6 models. This study uses the general model with two
levels, which is composed by two sub-models so-called Level 1 and Level 2
(Raudenbush & Bryk, 2002) in order to estimate the educational production function for
each period under analysis. In this model, the data referring to each student (i) are
contained within IES representing (j) groups of students. The Level 1 model reproduces
the relationship between an explanatory variable (Xij) with a dependent variable (Tij) and
the Level 2 Model captures the influence from IES factors. Formally, there are i=1, .., n,
units of the Level 1 within j = 1, ..., J units of the Level 2. In Eq. (1) and (2) we to show
the general model equations of two levels, with the use of the notation by Raudenbush
and Bryk (2002) and the framework of the educational production function (Todd &
Wolpin, 2003; Harris, 2010; Hanushek & Woessmann, 2011):
Level 1 (students) rXRPFAT ijpijpjijjijjijjijjjij ...43210
, (1)
Level 2 (IES) uW...IRP qjhjqhj33qj22qj11q0qqj . (2)
The terms β0j, ... βpj are coefficients of the Level 1 and the terms γq0, ... yqh are
coefficients of the Level 2, which can be understood as fixed effects. The term Xij is the
exogenous variable of the Level 1, which represents the students’ individual skills (Aij),
their personal characteristics, their family background (Fij), possible effects from the
pairs (Pij) and school resources (Rij). The term Wj represents the exogenous variable of
the Level 2, which may be linked to effects of the pairs (P1j), to school resources (R2j)
and to institutional peculiarities of either school and education system (I3j). There are p
exogenous variables (Xij ) for Level 1 and h explanatory variables (Wj) for Level 2. The
equations present intercept and slope grade random (βqj) for all p exogenous variables
(X i j) Level 1, explained by h variables (Wj) Level 2. The term ri j are the random effects
of the Level 1and in the case of Level 2 it is represented by the term µq j. The terms of
6 The other models are described in: Raudenbush; Bryk (2002), Raudenbush; Bryk; Congdon (2007) and
Rabe-Hensketh; Skrondal (2008).
17
errors rij and µq j must be mutually independent with normal distribution and zero mean.
The variance (σ²) Level 1 (within groups) is represented by the term rij. The variance
and covariance (τ00, τ01) of the components Level 2, also so-called variance between
groups, refer to the term µq j referring to group level.
The substitution of the Eq. (1) in Eq. (2) results the general model given by Eq. (3):
rXuuXWWXT ij
p
1qqijqjj0
p
1qqijkjqk
h
1k
h
1kkjk0
p
1qqij0q00ij
, (3)
In Eq. (3), the term Tij is the score of the student (i) enrolled at IES (j) in the
Exam (Provão 2002/2003 and Enade 2006). The term Xqij represents the matrix of
exogenous variables to the score at Level 1. The terms µij and rij are vectors of i.i.d.
error terms, that assemble the unobservable factors affecting the score of the students in
IES (j). The intercept (γ00) refers to average of the students’ score minus the average
effects of the explanatory variables Xqij, together with the coefficient vector of the
exogenous variables WkJ, which may vary between individuals according to IES they
attended. The variance and covariance matrix of the model is represented for Level 1 by
the term σ² (variance between students) whereas for Level 2 of the IES they would be
represented by notations: var(uqj) = τ2q(q=1,..., p) e cov(uqj, ukj) = τqk(q, k= 1, ...., p).
As pointed out by Raudenbush and Bryk (2002), the terms level 1 βqj, that are the
slope degree, that is the slope parameters, can be treated at Level 2 as fixed coefficients,
nonrandom variable coefficients and random coefficients of models with random
intercepts and slope. The first option is the effect from explanatory variables (Xqij) to be
treated as constant between units of Level 2. In other words, as fixing the coefficients at
Level 1, implies that:
βqj = γq0 , (4)
where γq0 is a common effect from variables (Xqij) for each IES Level 2, that is, the
effect from coefficients (βqj) is fixed between units of Level 2. The second option would
be the slope of the term βqj in function of an average value of the term γq0 added with a
random effect of each unit from Level 2:
.u0qqj qj (5)
In this case, the term ßqj has random behavior. The third option assumes the
slope degree to depend on term Wj , and this implies that:
.uqjW0qqj j1q (6)
This third option delineates part of the variation in the slope of the term βq j to be
explained by exogenous variables (Wj) of the second level, but there is a random
18
component (uqj) that is the error term. In this case, the effect from term Wj is considered
in the model and the residual variation in term βq j which is Var (uqj) = τqj is
insignificant.
Raudenbush, Bryk and Congdon (2007) affirm that the dimension of the T
matrix depends on the number of the coefficients at Level 2, which are specified with
random variation. It is assumed that the set of the different explanatory variables of
Level 2 can be used in each equation (Q + 1) of the Model at Level 2.Although the
coefficients (γqo) Level 2 may vary according to IES attended by students, in this work
only the intercept Level 1 (β0j) presents random variation, according to IES. This means
to adopt the model that estimate different intercepts for IEs and for each explanatory
variable, either Level 1 and Level 2, for which different coefficients are separately
estimated for each IES.
Raudenbush and Bryk (2002) suggest the use of the maximum likelihood
method for unbalanced data, that is, when there are different sample sizes at Level 1 for
each organizational unit Level 2. In such cases, the integral maximum likelihood
method generates more consistent and efficient estimators. Based on this
recommendation, the maximum likelihood method, in full or integral concept, is used in
the present study in order to estimate the coefficients because unbalanced structure of
the micro data. According to Favero et al. (2009), this method estimates the variance
and covariance parameters and the fixed effect coefficients of Level 2 by maximizing
their joint probability.
Raudenbush and Bryk (2002) pointed out the need for performing some of the
tests: fixed effect coefficients, random coefficients of Level 1 and variance-covariance
for all components. The first test is carried out to verify whether there is intra-class
correlation as well as the occurrence of one more residual term. The accomplishment of
this test requires the estimation of the unconditional or null model, from which the
specification is given by Eq. (7) and (8):
ry ijj0ij , (7)
u j000j0 , (8)
This null model allows to calculate the variance of the Level 1 (σ²) or between
students and the variance of the Level 2 (τ0²). With the terms related to variances of the
Level 1 and Level 2, the intraclass correlation is calculated based on Eq. (9).
19
220
20
, (9)
In this expression, the coefficient (ρ) represents the proportion of the total
variance among groups (Level 2 units). It is expected the resulting coefficient (ρ) to be
different from zero. The intra-class correlation is called by Bryk and Raudenbush
(2002) as the cluster effect and it is applied only to models with random intercepts.
Another important assumption to be verified is whether there are significant differences
among the means of the groups (j). The null hypothesis is that no significant differences
occur among means of the groups (j). The F test is a tool to test it. To test the fixed
parameters (γ), the null hypothesis is: H0: γq j = 0.
The Wald-typed test is recommended by Raudenbush, Bryk and Congdon (2007)
to check the statistical significance of the estimated variances and covariances, from
which the structure enables to test more than one coefficient. Its null hypothesis is H0:
Cγ = 0, which considers the statistical distribution of χ². To test the hypothesis for
normality of the error terms uqj and rij, the results from Shapiro-Francia W' test, the
behavior and the parameters obtained by histogram of both residues are verified. In
addition to those tests, Bryk and Raudenbush (2002) suggest the calculation of an
auxiliary statistics so-called reduced proportion of the variance or explained variance,
which allows the comparison between the estimated variance of the null model of each
separate component with that one of another model. Thus, the specifications for
calculation of the proportion of the explained variance at Level 2 and Level 1 are given
by Eq. (10) and (11):
ˆ
ˆˆR 2
0
21
202
2
, (10)
ˆ
ˆˆR 2
0
21
202
1
, (11)
With equation 10, the proportion of the explained variation Level2 ( R22
) is
calculated. The result from equation 11 is the proportion of the variance explained at
Level 1 ( R21
). The terms ̂ 20
and ̂ 20
are estimates of the residual variance of the null
model Level 2 and Level 1, respectively, whereas the terms ̂ 21
and ̂ 21
are estimates of
the residual variance of the conditional model to be compared with the null model Level
2 and Level 1, respectively.
20
According to Todd and Wolpin (2003), the models of the educational production
function are potentially subjected to endogeneity problems, especially due to absence of
unobservable inputs or explanatory variables omitted, such as students’ innate ability. In
this context, the recommendations by Rabe-Hensketh and Skrondal (2008) were
followed. According to those authors, the Hausman test should be used in order to
check the consistency of the estimators and the possible specification problems through
hypothesis for endogeny. The null hypothesis of the test allows to verify between
estimates of two distinct models which one is efficient, consistent and more adequately
specified. This test also indicates whether the most appropriate model is the one of fixed
effects or random effects. The adoption of the fixed effect model allows to use only the
information within groups. The authors suggest more accurate estimates of the
coefficients to be obtained, when information from both models within and among
groups are used. The Hausman test is used in case of the hierarchical models, especially
for evaluating the efficiency and consistency of the parameters estimated.
Taking into account this research objectives, the information concerning to
either large-scale tests and structure of the hierarchical linear model, the data referring
to each student (i) are contained within IES (j), that represent (j) groups of students at
first level and the data of the courses and institutions (j) as second level.
4. DATA AND RESULTS
4.1. Data
The main sources of the data used in this study were the micro data from Provão
(1997 to 2003), Enade (2006) and CES (2006 to 2009), provided by INEP. The data of
either Provão and Enade not allow for identification of the students. However, it is
possible to know the undergraduate course and IES to which the students were
affiliated. CES beheld data of the courses and institutions of the higher education
system and are divided into two files. The first file consists of data from each
undergraduate course. The second file provides information on institutions to which the
courses were affiliated. This form of the data availability allowed the information to be
considered with a hierarchical structure required to make estimates using HLM.
The micro data from Provão and Enade include the students’ responses to
socioeconomic questionnaires. There are information concerning to students, such as
21
personal characteristics, socioeconomic and cultural conditions, habits of reading and
study, as well as about courses such as facilities and resources available in institutions,
curricular structure, didactic-pedagogic organization and teachers’ performance. There
were issues common to all courses involved in the evaluation process and other ones
specific to the Accountancy course. There are more than 100 questions (INEP, 2004;
2007a). On exam day, the students also express their perception in relation to exam: the
resolution difficulty, extension, time sufficiency, statement of questions, personal
problems to resolve it, and whether the content was charged during course.
The analysis universe were the students graduating in Accountancy courses, who
attended and participated in the exams. In 2002, a total of 22,694 students were present;
in 2003, 23,495; and in 2006, 19,633. It was decided to remove from database those
students whose exam was not answered, in order to avoid the zero scores from
distorting the results, since it is possible the profile of the students who delivered the
exam in blank to be different from students who resolve the test, in terms of
commitment, dedication, etc. Then, the universe of analysis was reduced to 22,659,
23,451 and 19,497 graduating students, respectively. A total of 408 and 454 courses
participated of Provão in 2002 and 2003, respectively, as well as 811 courses in the case
of Enade/2006 (INEP, 2004; 2007a).
3.2. Descriptive analysis
In the hierarchical model, the dependent variable (Tij), which represents a proxy
of the academic performance of students graduating in Accountancy courses, always
varies at Level 1. The explanatory variables may vary among students’ observations
Level 1, among groups formed at Level 2 (of IES) and only in one of the levels (Rabe-
Hesketh & Skrondal, 2008). For example, the variable of gender varies among students,
whereas the variable indicating the type of administrative category to which the student
is affiliated (e.g., College) is constant among students and varies among IES. These
distinct variations entre among explanatory variables allow to calculate values with
central tendency and general dispersion, as considering each level.
Rabe-Hesketh and Skrondal (2008) recommend to calculate three standard
deviations for each variable. The first one is the overall or global standard deviation,
defined as the square root of the mean squared deviation of the observations, in relation
to general average. The standard deviation between groups is the square root of the
22
squared mean deviation of the group mean, in relation to general average. And the
standard deviation within is the square root of the mean squared deviation of the
observations, in relation to means of the groups. Three different maximum and
minimum values are also described: general, between and within. In addition, the
following values are presented: the total number of observations (N), the number (n) of
groups (j) formed and the average number of students (i) in each IES, from which the
symbol is reported in Tables 3, A1 and A2 is T-bar (Rabe-Hesketh & Skrondal, 2008).
Table 4 presents the basic statistics of the dependent variable, overall score. The
scale used by INEP for general score in both tests ranges from 0 to 100. The general
average scores obtained by students in the area were 32.1 in 2002 and 2003, and 33.9 in
2006, and 34.4 in 2009. The overall standard deviation of the general average obtained
in Enade decreased as compared to 2002 and increased from 2003 to 2006. The standard
deviations within and between allowed to verify that variation of the mean within IES
was higher, that is, the obtained average varied more because the different scores
achieved by students than among average scores obtained by IES. Although considering
the differences between form and content of the exams, it is observed the means of the
general score among different samples to be similar, since they remained around 32.1,
in 2002 and 2003 and 33.90 in 2006, and 34.4 in 2009.
Table 4
Values of the central tendency and dispersion in Provão 2002-2003 and Enade 2006-2009 of
Accountancy
Scores of the Provões and Enade in the Accountancy Courses
2002 2003 2006 2009
Student Numbers(N) 22,659 23,451 19,497 32,252
Groups (IES) (n) 343 379 523 628
T-bar 66.1 61.9 37.3 51.4
General mean 32.11 32.06 33.90 34.44
Median 30.5 31 33.4 33,2
Mode 21 25,5 26,6 26,6
General Standard deviation 11.27 9.45 10.10 13.46
Standard deviation between 4.51 4.34 4.16 6.03
Standard deviation within 10.17 8.60 9.22 12.265
General minimum 4 0 1,8 0.90
Mínimum between 20.33 22.32 6.4 4.2
Minimum within -1.29 -5.11 -5.05 -7.46
General Maximum 88.5 76 77.8 88.9
Maximum between 51.18 64 47.4 61.1
Maximum within 83.72 77.27 72.58 89.2
SOURCE: National Micro data of either National Exam of Courses (ENC/Provão) 2002 and 2003 and
Student Performance Exam (Enade) 2006 and 2009.
23
The micro data contain variables from which the level of measurement was
nominal and ordinal and at ratio scale. For cases of nominal and ordinal scale, some
procedures were adopted to represent them as numerical value. This codification of the
data in numerical measurement took into account the characteristics of the tools applied
to students by INEP. The main procedure was to construct a series of dummy variables.
It should be emphasized that some characteristics of either teachers and
educational process were collected through students, by using the variables estapr1,
dmcont1, mat1, aulexp1, csi1, pltr17. It was decided to allocate those characteristics at
level of the institutions because, although measuring the student’ perception, these
variables may be capturing choices made by managers of the institutions.
The exogenous variables used in this study were chosen with basis on the
constructs of the educational production function, the findings of the literature and the
availability of the database. For example, the choice of the variables that measure the
construct effect of the pairs was based on definition by Sacerdote (2011)8. Those
constructs or underlying attributes were used to represent a set of measurements, that
was generated by means of the variable group.
The basic statistics of the exogenous variables, as following the recommendation
by Rabe-Hesketh and Skrondal (2008), are reported in Tables A1 and A2, in Appendix.
Table A1 presents the descriptive statistics of the exogenous variables of first level. The
variables nidade, dsex, detnia and dded9 represent the construct that objectively measure
the personal characteristics of the students (Fij). In all years, the values of the means of
those variables pointed out that, among students approximately half are masculine
7 Level 2 independent variables: estapr1 = Ratio of responses given by students regarding IES to have
teachers who conducted research activities as learning strategy; mat1 = Proportion of responses given by
students regarding IES to have teachers who indicated the use of books, copies of book chapters and
handouts; aulexp1 = Proportion of responses given by students regarding IES to have the majority of
teachers whose teaching practice are predominantly lectures; csi1 = Proportion of responses given by
students in relation to IES that provided the knowledge on accounting information system; pltr1 =
Proportion of responses given by students regarding IES that the issue ‘tax planning’ was considered in
the course.
8Sacerdote (2011) defined the effect from pairs as any externality regarding the history and current
behaviors of the pairs or results that may affect its performance. The author used this ample definition,
but it limits the pair effects for externalities concerning to pairs or colleagues, family history or ongoing
actions.
9 Level 1 independent variables: nidade = Current age informed; dsex = 1 female, 0 masculine; detnia = 1
whites and yellows, 0 blacks and browns; dded = 1 spent at least one hour studying beyond classroom, 0
contrary case.
24
gender, more than half reported to be White and/or Yellow and spent at least one hour
daily to study. In each sample, the average age of students ranged from 28 to 29 years.
The standard deviations of the variables listed in Table A1 rather suggest that dispersion
around mean to have no significant changes over the years. The values of the standard
deviations within indicate the variation of those variables to be higher within than
among IES.
It is assumed the effect of the pairs (Pij) in the context of the students can be
apprehended by variables dfilhos, and dirmão decivil10. Table A1 shows that more than
half of the total students declared to be single, childless and over 90% reported to have
siblings. The dispersion of the variables reported in Table A1 was also higher within
IES than among IES.
The family and socioeconomic factors (Fij) were measured by means of the
exogenous variables despai1, desmãe, enmdpúb11 and the family income. The means of
those variables, shown in Table A1, suggest that over 80% students reported to be
children of parents who have not attained higher education, and more than half attended
the high school integrally or mostly in public school. The income ranges 6 and 7 were
reported only for 2006 in Table A1 because they were added to socioeconomic
questionnaire this year. During the periods under analysis, the mean increased in the
income ranges 1, 3 and 5. In the income range 2, the mean decreased and there was no
concentration in any income range. It was observed that the values of the means of the
income ranges 1, 3, 2 and 4 from 2003 to 2006 have changed significantly. In Table 2,
the values of the standard deviations within and between show the variations to be
greater within IES than among IES.
10 Level 1 independent variables: = dfilhos = 1 has son/daughter (s), 0 otherwise; dirmão = 1 if has sibling
(s), 0 otherwise; decivil = 1 unmarried, 0 married, separated, widowed and others.
11 Level 1 independent variables: = despai1 father with higher education, 0 otherwise; desmãe = 1 mother
with higher education, 0 otherwise; enmdpúb = 1 attended the high school in the public school, 0
otherwise. income_1 = 1 family income range up to R$ 720.00, and 0 otherwise; income _2 = 1 family
income range from R$ 721.00 to R $ 2,400.00, 0 otherwise; income_3 = 1 family income range from R$
2,401.00 to R$ 4,800.00, 0 otherwise; income_4 = 1 family income range from R$ 4,801.00 to R$
7,200.00, 0 otherwise; income 5 = 1 family income range more than R$ 7,201.00. Values of the level 1
independent variables Enade 2006: 0 otherwise. income_1 = 1 family income range up to three minimum
wages, and 0 otherwise; income _2 = 1 family income range more than 3 to 5 minimum wages; 0
otherwise; income_3 = 1 family income range more than 5 up to 10 minimum wages, 0 otherwise; income
4 = 1 family income range over 10 to 15 minimum wages, 0 otherwise; income_5 = 1 family income
range over 15 to 20 minimum wages, 0 otherwise; income_6 = 1 family income range more than 20 to 30
minimum wages, 0 otherwise; income _7 = 1 family income range more than 30 minimum wages.
25
In general, the option to participate in extension and scientific initiation
activities arises within IES and the students choose it or not. The way this choice uses
school resources (Rij) was captured through variables reported in Table A1. In this
context, it should be noted that the dispersion of those variables within IES was higher
among IES. It is emphasized that the average of the variable measuring the participation
into scientific initiation increased from 0.06 in 2002 to 0.27 in 2006. This might be a
reflection of the increase in supply of this activity by IES, which generated greater
student participation. I is noted the mean and the dispersion values referring to
extension activity were similar over the years.
Table A2 presents the descriptive statistics of the second-leveled exogenous
variables. The within standard deviation of the variables measured at second level is
zero. Because construction, those variables vary only among IES and not among
students. Therefore, they are constant at Level 1 (Rabe-Hesketh & Skrondal, 2008).
Table A2 shows the variables referring to main characteristics of the teachers, as
considered as school resource (R2J), and the organizational form of either IES and
educational system (I3j). The mean proportions of teachers with master and doctorate
titles, full-time working hours and content domain increased over the period from 2002
to 2006. The variance of the md variable referring to proportion of the teachers with
master and doctorate degrees in the institutions decreased from 0.05 to 0.04 over the
period from 2002 to 2003. It is possible the changes in those values to be related with
the need for institutions to fulfill the percent teachers with those titles determined by
MEC. The behavior of both average ratio and standard deviation among IES of the
academic organization (dorgac) was heterogeneous. The increase in values of the
variable dorgac is related to an increased number of Universities and University centers
throughout the years. It is observed the mean and standard deviation between IES and
other variables to be similar in this period.
The statistics related to main aspects of the educational process, that were
adopted in the courses, are reported in Table A2. The values of the standard deviations
among groups of those variables were generally low. The average proportion of either
teachers who conducted research activities as teaching and learning strategy and
teachers who used lecture as teaching practice decreased over the period from 2003 to
2006. In 2006, the dispersion among IES concerning to have lecture was nonexistent
The values of the variables csi1 and pltr1 showed that the contents of the accountable
26
information system and tributary planning were included in the courses, either in 2002
and 2003. The statistics of the other variables were similar over this period.
Table A2 presents the descriptive statistics of the environmental or context
variables so-called effects of the pairs (P1j). The results of either means and standard
deviations among groups showed homogeneous character. Despite differences in
number of the observations, groups (IES) and mean number of observations within
groups (T-bar), there were little changes during the period under study.
According to data presented in Tables 3 and 4, there is high probability the most
students to compose the first generation of the family, as attending an undergraduate
course, since 80% students were sons/daughters of parents with no training in higher
education. Other uniformities were observed, such as similarity of the distribution by
gender and by frequent use of the textbooks and/or manuals and handouts as didactic
material by teachers of the course. Moreover, according to the report by INEP, the
students’ perception, certain basic contents covered on the exam were given, but the
given approach was different from the one charged (INEP, 2004). Concerning to
teachers, according to the students' perceptions of the area, the most frequent
characteristics were content domain and the use of research activity as learning strategy.
Another point was the high use of lectures or lecture with student participation, such as
technical education, with approximately 80% total responses. On average: 60% teachers
worked as hourly employees and 37% were still experts, despite the increased number
of the master and doctor teachers.
3.2. Results from estimations of the Hierarchical Linear Models
A production function of the educational course in Accounting was estimated for
each year using hierarchical linear models, with the maximum likelihood method and
based on the previously established hypotheses. The econometric procedure was done
in software Stata, version 9.1. The order for presentation of the estimates in Table 5 was
based on the literature of hierarchical models (Raudenbush & Bryk, 2002; Rabe-
Hesketh & Skrondal, 2008; Fávero et al., 2009).
The first estimate is done with the unconditional or null model without
explanatory variables. According to Fávero et al. (2009), the estimation of this model
allows to verify the following hypotheses: (1) there is significant variability in
27
performance among students of the same IES; and (2) there is significant variability in
performance among students from different IES. These assumptions allowed to evaluate
whether there were differences in performance among students and among IES.
The results from estimates of the null or unconditional model referring to Provão
2002/2003 and Enade 2006/2009 are presented in Table 5. Those estimates allowed to
calculate the IES average performance in tests (β0j) as a function of the overall average
of all IES (γ00), added with a random component (u0j). Initially, it is considered the IES
average performance did not vary from institution to institution and was treated as
constant. The results showed the expected average performance of the students, that is,
31.29 in Provão 2002, 31.67 in Provão 2003 and 33.25 in Enade/2006 and 33.94 in
Enade/2009. The residual standard deviations of Level 1 were 10.25 in Provão 2002,
8.67 in Provão 2003 and 9.35 in Enade/2006 and 12.38 in Enade/2009. The standard
deviations of the random intercepts of Level 2 were 4.26 in Provão 2002, 3.62 in Provão
2003 and 3.60 in Enade/2006 and 5.16 in Enade/2009.
The variance of the overall score in Provão/2002 among students was 104.96
(Level 1) and among institutions was 18.15 (Level 2). The sum of those variances
generated a total value of 123.11. In Provão 2003, the calculated variance for Level 1
was 75.21 whereas for Level 2 was 13.11, therefore its sum corresponds to total 88.32.
The variance of the overall score in Enade/2006 among students was 87.39 (Level 1)
and the variance among institutions was 12.97 (Level 2), and their sum equals a total
variance of 100.36. The variance of the overall score in Enade/2009 among students
was 153.44 (Level 1) and the variance among institutions was 26.67 (Level 2), and their
sum equals a total variance of 180.11. These results indicate that most variation in
overall score in each year occurred in Level 1, that is among students.
Those proportions of the level variances allowed to calculate the intraclass
variance, which indicates the proportion of the total variance due to IES groups as
defined in Level 2 (Raudenbush & Bryk, 2002; Rabe-Hesketh & Skrondal, 2008; Fávero
et al., 2009). The values of the intraclass correlation coefficients (ρ) of the null model in
Provão/2002 was 0.147, in Provão/2003 was 0.148 and in Enade/2006 was 0.129 and in
Enade/2009 was 0.148. These values suggest that part from proportion of the total
variance can be attributed to IES peculiarities. Thus, it is possible to include specific
variables of the institution level in the model, that is, at second level. This indicates
28
adequacy of the hierarchical models for estimating the educational production function
of the Accountancy area. Besides, it shows that most of the variation of the overall score
in Provão/2002, Provão/2003, Enade/2006 and Enade/2009 occurred more within IES
than among institutions. It is observed the value of the intraclass correlation of the
unconditional model of 2003 and 2002 to be higher than the estimated value in 2006
and 2009.
Based on results, we did a new estimate of the full model. In this procedure, all
proposed explanatory variables were included. The objective is to understand what
characteristics of the students and institutions had significant effect on the dependent
variable academic achievement. The results from those estimations for each year are
reported in Table 5.
Table 5
Estimates of the two-level Linear Hierarchical Model of the Accountancy students’ academic
achievement in Provão/2002/3003 and Enade 2006/2009.
2002 2003 2006 2009 2002 2003 2006 2009
Model Model
Fixed Effects Null Null Null Null Full Full Full Full
Nidade -0.00775 0.0269** -0.0573*** -0.0744***
(0.0153) (0.0118) (0.0142) (0.0142)
Dsex -4.095*** -2.528*** -1.657*** -2.969***
(0.159) (0.125) (0.160) (0.192)
Decivil -1.149*** -1.013*** -1.177*** -0.0953
(0.216) (0.174) (0.212) (0.214)
Dirmão 0.906*** 0.0994 -0.0440 -
(0.351) (0.290) (0.383) -
dfilhos 0.359 0.283 -0.0166 -
(0.245) (0.191) (0.237) -
detnia 0.498** 0.250* 0.121 0.309
(0.199) (0.151) (0.187) (0.210)
income _2 2.336*** 1.806*** 1.829*** 0.369
(0.291) (0.197) (0.234) (0.375)
income 3 4.076*** 3.148*** 2.907*** -0.288
(0.317) (0.227) (0.229) (0.389)
income_4 5.075*** 3.246*** 3.601*** -0.340
(0.379) (0.302) (0.299) (0.424)
income_5 5.333*** 4.240*** 4.742*** -0.302
(0.717) (0.385) (0.421) (0.418)
income_6 - - 5.189*** -0.329
- - (0.547) (0.487)
income_7 - - 5.246*** 0.373
- - (0.640) (0.985)
despai -0.253 -0.389** 0.160 -0.0633
(0.251) (0.197) (0.274) (0.344)
desmãe -0.419 -0.139 -0.423 -0.407
(0.268) (0.204) (0.271) (0.317)
29
2002 2003 2006 2009 2002 2003 2006 2009
Modelo Modelo
Null Null Null Null Full Full Full Full
Fixed Effects Coef. Coef. Coef. Coef. Coef. Coef. Coef. Coef.
enmdpúb 0.866*** 0.850*** 0.383** -0.0623
(0.180) (0.142) (0.191) (0.254)
dded -
1.687***
1.807*** 1.406*** 0.433
(0.326) (0.193) (0.246) (0.281)
dexten 0.451* -0.300 -0.199 0.145
(0.264) (0.208) (0.246) (0.254)
dic 0.193 -0.809*** -0.492** 0.150
(0.339) (0.153) (0.192) (0.223)
dorgac 0.301 0.835** 0.813** 1.992***
(0.697) (0.409) (0.363) (0.489)
dcatad -1.367 -1.783*** -1.131* -
3.367***
(0.997) (0.661) (0.585) (0.686)
Md 0.742 3.088*** 3.716*** 4.345***
(1.184) (1.185) (1.006) (0.711)
v_intg -2.258 1.683* 4.280*** -0.146
(1.401) (1.000) (0.917) (0.752)
aulexp1 12.88*** 6.698*** 741.2*** -
(2.165) (1.771) (163.2) -
estapr1 0.0362 5.835** 6.251*** -3.092
(5.216) (2.816) (1.886) (2.205)
mat1 9.605*** 3.535 -1.406 0.487
(3.370) (3.017) (2.646) (2.503)
dmcont1 5.895** 5.951*** 5.918*** 0.487
(2.792) (2.200) (2.052) (2.503)
csi1 2.675 1.657 - -
(2.043) (1.614) - -
pltr1 0.432 -1.019 - -
(2.505) (1.946) - -
espai1 5.170 5.484** 5.995*** 0.447
(3.494) (2.542) (2.281) (2.241)
esmãe1 4.222 -8.659*** -
6.569***
-2.358
(4.295) (2.793) (2.436) (2.043)
Midade 0.0339 0.00762 -0.0519 -0.165***
(0.103) (0.0734) (0.0610) (0.0713)
Constant 31.29*** 31.67*** 33.25*** 33.94*** 10.71* 11.93*** 21.68*** 42.89***
(0.247) (0.201) (0.176) (0.226) (6.368) (4.194) (3.591) (2.246)
Random Effects
sd(_cons) 4.260*** 3.621*** 3.602*** 5.164*** 3.619*** 2.773*** 2.380*** 4.219***
(0.183) (0.151) (0.135) (0.173) (0.183) (0.136) (0.127) (0.179)
sd(Residual) 10.25*** 8.672*** 9.348*** 12.38*** 9.954*** 8.482*** 9.253*** 12.253***
(0.0485) (0.0404) (0.0480) (0.492) (0.0549) (0.0432) (0.0554) (0.0554)
Derivated Estimates
R² 0.00 0.00 0.0000 0.00 0.00981 0.00810 0.00705 0.00308
Ρ 0.147 0.148 0.129 0.148 0.117 0.0966 0.0620 0.1061
Prob >= chibar2 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Variance of the Level 2 18.15 13.112 12.974 26.671 13.10 7.680 5.664 17.798
variance of the Level 1 104.96 75.204 87.385 153.443 99.08 71.944 85.618 149.983
Total Variance 123.11 88.315 100.359 180.114 112.18 79.634 91.282 167.782
30
Derivated Estimates 2002 2003 2006 2009 2002 2003 2006 2009
R22
0.2783 0.4135 0.5634 0.00308
R21
0.0560 0.0433 0.0202 0.00308
Proportion of the Total Variance explained 0.0888 0.0983 0.0905 0.0735
Log likelihood -85264 -84352 -72752 -12757 -62415 -70833 -5267 -68662
Wald chi2(0) . - - 1295 1281 871.9 447.3
Prob > chi2 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Hausman chi2 (16)24.64 (16) 55.38 (17) 69.99 (24) 26.,54
Hausman Prob>chi2 = 0.0765 0.0000 0.0000 0.3263
Number of observations 22,659 23,451 19,497 32,252 16,719 19,651 14,330 17,266
Number of Groups 343 379 523 628 277 339 408 604
Obs, min, by/groups 4 1 1 1 4 1 1 1
Obs, med, by/groups 66.06 61.88 37.28 51.4 60.36 57.97 35.12 28.6
Obs, max, by/groups 357 402 216 482 337 376 198 323
SOURCE: National Micro data of either National Exam of Courses (ENC/Provão) 2002 and 2003
and.Student Performance Exam (Enade) – 2006 and 2009.
In Table 5, the results show the variables dsex and dded are significant at 1% in
either Provão 2002 and 2003 and Enade 2006. The sign of the dsex variable coefficient
was negative in all years. This signal indicates that female students, controlled by the
other factors, tended to present worse academic performance than men. This result
corroborates the findings by Diaz (2007) in relation to courses of administration, law
and civil engineering. The effect from the dded variable on Provão 2002 was also
negative, although 94% students in 2002 have answered they dedicated at least one
additional hour to studies, besides classroom. In Provão 2003 and Enade 2006, the sign
of the dded variable coefficient indicated a positive relationship with students’
performance. It was observed that older students tended to obtain higher performance,
as compared with the younger ones in Provão 2003. However, the opposite occurred in
Enade 2006 and 2009, since the coefficient sign of the variable nidade was negative.
The coefficient of the variable detnia was positive in years under observation. However,
the magnitude of its effect was low and only statistically significant at 5% in Provão
2002 and 10% in Provão 2003. This represents that students who declared to be white or
yellow, in Provão 2002 and 2003, tended to achieve higher academic performance than
other ethnicities.
31
It is observed that the effect produced by the fact the student to be single was
negative in Provões (2002 and 2003) and Enade (2006). However, in the estimated
model of the Provão 2002, there is a possibility for this negative effect to be partially
compensated among students who have siblings.
The coefficients of the socioeconomic and households’ factors were positive in
all regressions, except in 2009. It is observed the coefficients of the income ranges to be
higher for higher levels. Moreover, a positive effect upon student performance was
observed, when the student had studied totally or part of the high school in public
school. The overall results also allow to infer that there is a positive association between
family background variables and the student’ academic performance.
The variables referring to educational levels of the mother and father were
statistically significant only in some models. In Provão 2003, the effect from despai
variable coefficient was negative, but its magnitude is low. In contrast, the effect from
the espail variable coefficient was positive in Provão 2003 and Enade 2006. Diaz
(2007) obtained similar results for father's educational level, in courses of
administration, law and civil engineering. However, the author found negative
coefficients in both Provão 2003 and Enade 2006 for the context variable esmãe1 which
captures the proportion of mothers with higher education in IES. These results may be
be related from high frequency (about 85%) of students whose father had no higher
education at the time (espai1) and around 88% mothers had no higher education at the
time, in all years. These results indicate that students have underperformed in IES, with
low proportion of students whose mothers have higher education. However, this effect
may be compensated in IES with higher proportion of students whose parents have
higher education. Besides, this high percentage of students with parents without higher
education rather suggests they may have been the first individuals from one family
generation to have access to higher education.
After analyzing the characteristics of the students, the next step was to verify the
influence from the resources provided by IES, according to students’ responses. For the
estimated model of Provão 2002, only the variable dexten was significant and positive
at 10%. Despite low magnitude of the coefficient, this result suggests the participation
of the students in extension activities to improve their performance. The student
participation in activities related to undergraduate research (ddic) was negatively
32
associated with performance of the students in both Provão 2003 and Enade 2006, after
its effect to be controlled by other factors. The negative effect from this coefficient
should be cautiously interpreted, since the frequency of the students participating in the
undergraduate research activity was low, that is approximately 25%, whereas the
variance among IES was 0.03 in 2003 and 27% in 2006.
Among variables referring to typical resources of the course educational process,
it is observed the variable mat1 to be significant and positive only in 2002. This showed
the improved performance of the students in IES, where the proportion of teachers who
use more frequently the textbooks, manuals, handouts, summaries, copies of sections or
chapters of books were higher. The variables aulexp1 (all years) and estapr1 (2003 and
2006) were also statistically significant. The coefficients of the variable
aulexp1indicated that, if all other variables were kept constant, an increase in proportion
of teachers who use lecture in IES would be associated with students’ improved
performance in either Provões (2002 and 2003) and Enade ( 2006). The interpretation of
the value 741,2 of the coefficient of the variable aulexp1, from which the standard error
was 163,2, deserves caution. This high value of the standard error can be explained by
the lacked dispersion of this variable among IES in 2006, since the standard deviation
among institutions was 0,001 and the average was 0.002. The sign of the estapr1
variable coefficient indicated the increase in students’ performance in Provão 2003 and
Enade 2006 would be positively related with the increase in proportion of teachers who
requested the accomplishment of the research activity as learning strategy.
The coefficient of the variable dmcont1, which aimed to capture, under students’
perception, the content domain by the faculty in IES, was significant and had positive
sign in all regressions, except in 2009. This result indicates that students’ performance
tended to be higher among IES with higher frequency of teachers who demonstrated the
mastery of content. The variables md, v_integ, dorgac and dcatad were statistically
significant in estimates of the HLM models in Provão 2003 and Enade 2006. Among
these variables, only v_integ coefficient wasn’t significant in 2009. The coefficients of
md and v_integ can be interpreted together, since they represent characteristics of the
faculty affiliated to IES. The results show that students’ performance tend to be
significantly positive in IES which have higher proportion of teachers with full-time
working hours as well as master and doctor degree. The effect on performance was also
positive, when the student was affiliated to universities or university centers (dorgac),
33
as other factors are held constant. This positive effect is reduced among students
affiliated to particular IES.
The diagnosis of the distribution of the residuals derived from full models each
year was based on either histogram of the residues and the test Shapiro-Francia W. The
results do not reject the multivariate distribution of the residues generated by fixed and
random effects from models to be normal. It was also verified whether the set of
estimators of the full model was not efficient, by using the Hausman test. The obtained
result indicates the set of the full model specifications to be effective, in 2002. In 2009,
the result was similar to 2002, with chi2 of the 26.54 and Prob> chi2 of the 0.3263.
However, the result from Hausman test, reported in Table 2, brought evidence of
endogeneity12 for estimation of the full model 2003 and Enade 2006. Despite the result
from Hausman test, it was opted to present the model with fixed and random effects in
order to report the information from the estimates accomplished.
In the full models 2002, 2003, 2006 and 2009, there was reduction in either total
variance and intraclass correlation (ρ), as compared with the value obtained by
unconditional models. Those results suggest the correlation among students of the same
IES to be reduced, as corroborated by controls based on individual and institutional
aspects. This reduction also occurred in relation to variances achieved at Level 1 and
Level 2. Those decreases in the values of the variances and intraclass correlation (ρ)
between unconditional model and full model rather indicated the variables included in
the final model to explain part of the variance occurring among students and among
institutions. Based on those results, it is possible to accept the hypotheses that there are:
(3) students’ characteristics explaining the variability in performance within the same
IES; (4) students’ characteristics explaining the variability in performance among IES;
and (5) course characteristics explaining the differences in performance among students.
5. FINAL COMMENTS
The objective of this study was to analyze the effect from individual and
institutional characteristics upon students’ academic performance in Accountancy, in
Brazil, through the results achieved in either ENC-Provão 2002 and 2003 and Enade-
2006/2009. The main empirical evidences were obtained through estimates of the
educational production function, as employing Hierarchical Linear Models.
34
Based on comparison of the estimates obtained with full models and those
obtained with unconditional model (reported in Table 5), it was observed a reduction of
the total variance values. This reduction occurred at both levels, but the proportion of
the explained variance was higher at Level 2. This result indicates the effect from all
variables related to IES upon variation in academic performance of the students
graduating in the area, in years 2002, 2003, 2006 and 2009, to be significant in
comparison to proportion of the variance explained by the set of variables Level 1
associated with the level of the student.
The main results suggest a significant association between the academic
performance of students graduating in Accountancy in years 2002, 2003 and 2006 and
certain characteristics, own or from family, such as gender, hours dedicated to studies,
family income range, have studied in public high school. In IES ambit, it is emphasized
to have had teachers with content domain besides adopting the lecture as predominant
teaching practice. In the category ‘effects from pairs’, in all years under observation, a
negative and significant relationship of the student’ academic performance was found
because being single, except in 2009. The findings indicate that the effect on students’
performance tended to be positive in the institutions from which the teachers had master
or doctoral titles and full-day working hours (40h), or exclusive dedication to teaching,
who more frequently used the research as educational strategy in 2003 and 2006.
The evidences concerning to parental education and the student participated in
the undergraduate research activity rather suggest the need for more detailed exam. In
the first case, the different signs of the coefficients of those variables may come from
low mean proportion of students whose mothers and fathers have higher education
lower than 20% in all years under analysis. In the second case, the effect was negative.
However, it was expected the involvement of the students in research activities to
support the improvement of their academic performance. Therefore, it seems necessary
to investigate more closely this group of students in order to verify whether the time
devoted to this activity is negatively interfering on their performance or there are other
factors explaining the aggravation of the results.
Despite the magnitude of the coefficients have been low, the negative effect on
performance of the students affiliated to private IES and the positive effect on
performance of the students affiliated to Universities may be related to profile of the
35
student selected by IES. There is possibility that the selection process used by
Universities has selected those students whose academic profile is more appropriate
than the one of the private IES.
It was observed that most variables showed to be relevant to explain the
academic performance of the students graduating in Accountancy courses, particularly
in 2003 and 2006. However, the low proportion of the explained total variance, that is
0,088 in 2002, 0,098 in 2003 and 0,091 in 2006, and the results from Hausman tests that
showed endogenous problems for estimates of the hierarchical linear model in 2003 and
2006 eventually suggest the specification of the model could be improved through
inclusion of other explanatory variables. A possible alternative to eliminate endogeny
could be the use of instrumental variables, as mentioned by Ludwig and Bassi (1999)
and Todd and Wolpin (2003). It is possible to construct other variables based on
available databases or to improve the data collection tools in order to make better the
available measures or to insert, for example, the variable “classroom size” suggested by
Ludwig and Bassi (1999 ).
This study showed evidence for possible indicators and their effects on academic
performance of the students graduating in Accountancy, in Brazil. In addition, it directs
the attention to reflect on quality of the training that students are receiving, since those
students will have great insertion and professional performance in the labor market in
Brazil.
References
Albernaz, A., Ferreira, F. H. G., & Franco, C. (2002). Qualidade e Equidade na
Educação Fundamental Brasileiro. Pesquisa e Planejamento Econômico (PPE),
32(3), 453-476.
Andrade, J. X., & Corrar, L. J. (2008). Condicionantes do desempenho dos estudantes
de contabilidade: evidências empíricas de natureza acadêmica, demográfica e
econômica. Revista de Contabilidade da UFBA, 1, 62-74.
Betts, J. R., & Morell, D. (1999). The determinants of undergraduate grade point
average: The relative importance of family background, high school resources, and
peer group effects. Journal of Human Resources, 34(2), 268–293.
Conselho Federal de Contabilidade (2010a). Accessed from:
http://www.cfc.org.br/conteudo.aspx?codMenu=64 , 2010.
Conselho Federal de Contabilidade (2010). Accessed from:
http://www.cfc.org.br/conteudo.aspx?codMenu=67&codConteudo=5702
, 2011.
36
Cohn, E., Cohn, S., Balch, D. C., & Bradley JR., J. (2004). Determinants of
undergraduate GPAs: SAT scores, high-school GPA and high-school rank. Economics
of Education Review, 23, 577–586.
Crespo, A.; & Reis, M. C. (2009). Sheepskin effects and the relationship between
earnings and education: analyzing their evolution over time in Brazil. Revista
Brasileira de Economia, 63( 3), 209-231.
Cruz, C. V. O. A., Corrar, L. J., & Slomski, V. (2008). A docência e o desempenho dos
estudantes dos cursos de graduação em contabilidade no Brasil. Contabilidade
Vista & Revista, 19, 15-37.
Diaz, M. D. M. (2007). Efetividade no ensino superior brasileiro: aplicação de modelos
multinível à análise dos resultados do exame nacional de cursos. Revista
EconomiA, 8(1), p. 93-120.
Fávero, L. P. L., Belfiore, P. P., Chan, B. L., & Silva, F. L. da (2009). Análise de dados:
modelagem multivariada para tomada de decisões. Rio de Janeiro: Elsevier.
Franco, A. M. de P.; & Menezes-Filho, N. A. (2009). Os determinantes do aprendizado
com dados de um painel de escolas do SAEB. Anais do Encontro Nacional de
Economia, Foz do Iguaçu, PA, Brasil, 38.
Gracioso, A. (2006) Análise da Eficácia Escolar e do Efeito-Escola nos Cursos de
Administração de Empresas no Brasil. Tese de doutorado em Administração de
Empresas, Programa de Pós-Graduação em Administração de Empresas da Escola
de Administração de Empresas de São Paulo da Fundação Getúlio Vargas, São
Paulo, SP, Brasil.
Hanushek, E. A (1979). Conceptual and empirical issues in the estimation of
educational production functions. The Journal of Human Resources, 14 (3), 351-
388. Retrieved from https://www.jstor.org/stable/145575
Hanushek, E. A. Educational production functions (1987). In Psacharopoulos, G. (Ed.).
Economics of education research and studies (1nd ed., p. 33-42) New York:
Pegarmon Press.
Hanushek, E. A., & Woessmann, L. (2011). The economics of international differences
in educational achievement. In Hanushek, E., Machin, S., & Woessmann, L. (Eds.).
Handbook of the economics of education (1nd ed., 3 v., p. 89-200). Oxford (UK):
Elsevier Science.
Harris, D. N (2010). Education production functions: concepts. In Brewer, D. J., &
Mcewan, P. J. (Eds.). Economics of education (1nd ed., p. 127-131). Oxford (UK):
Elsevier Academic Press.
Horowitz, J. B., & Spector, L. (2005). Is there a difference between private and public
education on college performance? Economics of Education Review, 24, 189–195.
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2009).
Brasília, DF. Retrieved from http://portal.inep.gov.br/web/censo-da-educacao-
superior/evolucao-1980-a-2007
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2010).
Brasília, DF. Retrieved from http://portal.inep.gov.br/basica-levantamentos-
microdados
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2003).
Sistema Nacional de Avaliação da Educação Superior (Sinaes): bases para uma
proposta de avaliação da educação superior brasileira. Brasília, DF. Retrieved
from http://unifesp.br/reitoria/orgaos/comissoes/avaliacao/sinaes.pdf
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2004).
Relatório do Exame Nacional de Cursos 2003: Ciências Contábeis. (v. 6). Brasília,
37
DF. Retrieved from
http://www.inep.gov.br/superior/provao/diretrizes/2003/ciencias_contabeis.htm
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2007). Enade
– Resultados agregados. Brasília, DF. Retrieved from
http://www.inep.gov.br/superior/Enade/default.asp
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2007a).
Relatório Síntese: Ciências Contábeis. Brasília, DF. Retrieved from
http://www.inep.gov.br/superior/Enade/default.asp
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2011). Enade
2009 – Relatório de curso: Fundação Universidade Federal de Viçosa. Brasília,
DF. Retrieved from http://www.inep.gov.br/superior/Enade/default.asp
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2013). Enade
2012 - Relatório Síntese: Ciências Contábeis. Brasília, DF. Retrieved from
http://www.inep.gov.br/superior/Enade/default.asp
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2010).
Resumo Técnico: censo da educação superior 2009 – dados preliminares. Brasília,
DF. Retrieved from:
http://download.inep.gov.br/download/superior/censo/2009/resumo_tecnico2009.p
df
Ludwig, J., & Bassi, L. J. (1999). The puzzling case of school resources and student
achievement. Educational Evaluation and Policy Analysis, 21(4), 385-403.
Machado, A. F. et al. (2008). Qualidade do ensino em matemática: determinantes do
desempenho dos estudantes em escolas públicas estaduais mineiras. Revista
EconomiA, 9(1), 23-45.
Moreira, A. M. de A. (2010). Fatores Institucionais e Desempenho Acadêmico no
Enade: um estudo sobre os cursos de Biologia, Engenharia Civil, História e
Pedagogia. Tese de doutorado em Educação, Programa de Pós-Graduação em
Educação da Faculdade de Educação da Universidade de Brasília, Brasília, DF,
Brasil.
Rabe-Hesketh, S., & Skrondal, A. (2008). Multilevel and longitudinal using Stata (2 nd
ed.). College Statio, Texas: A Stata Press Publications.
Raudenbush, S., & Bryk, A. (2002). Hierarchical linear models: applications and data
analysis methods (2 nd ed.). Thousand Oaks: Sage Publications.
Raudenbush, S., Bryk, A., & Congdon, R. (2007). HLM 6: hierarchical linear and
nonlinear modeling. Suite: Scientific Software International, Inc..
Rezende, M. (2010). The effects of accountability on higher education. Economics of
Education Review, 29, 842–856.
Sacerdote, B. (2011). Peer effects in education: How might They work, How big are
They and How much do We know thus far? In Hanushek, E., Machin, S., &
Woessmann, L. (Eds.). Handbook of the economics of education (1nd ed., 3 v., p.
249-277). Oxford (UK): Elsevier Science.
Soares, J. F., Ribeiro, L. M., & Castro, C. de M. (2001). Valor agregado de instituições
de ensino superior em Minas Gerais para os cursos de Direito, Administração e
Engenharia Civil. Dados – Revista de Ciências Sociais, 44(2), 363-396. Retrieved
from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0011-
52582001000200005&lng=en&nrm=iso
Souza, E. S. de. (2008) ENADE 2006: determinantes do desempenho dos estudantes do
curso de Ciências Contábeis. Dissertação de mestrado em Ciências Contábeis,
Programa Multiinstitucional e Inter-Regional de Pós-Graduação em Ciências
Contábeis da UnB, UFPB, e UFRN, Brasília, DF, Brasil.
38
Todd, P. E., & Wolpin, K. I. On the Specification and Estimation of the Production
Function for Cognitive Achievement. The Economic Journal, 113(485), F3-F33.
Retrieved from: http://www.jstor.org/stable/3590137
APPENDIX - Table A1 Summary statistics of the exogenous variables of Provão 2002/2003 and Enade/2006 of Accountancy of Level 1 - Students.
Variables 2002 2003 2006
Mean SD Variance Min. Max. Observations Mean SD Variance Min. Max. Observations Mean SD Variance Min. Max. Observations
Construct: F – Personal characteristics
nidade
overall 27.88 6.59 43.43 0 75 N 22,659 28.68 6.81 46.38 0 75 N 23,451 29.04 7.35 54.02 10 87 N 19,497
between 2.56 6.55 22.95 39.89 n 343 2.64 6.97 23.13 42.5 n 379 2.94 8.64 22.96 48 n 523
within 6.27 39.31 0.67 76.23 T-bar 66.1 6.49 42.12 3.52 72.74 T-bar 61.9 6.88 47.33 7.16 87.32 T-bar 37.3
dsex
overall 0.51 0.50 0.25 0 1 N 22,659 0.49 0.50 0.25 0 1 N 23,451 0.53 0.50 0.25 0 1 N 19,497
between 0.11 0.01 0 0.82 n 343 0.13 0.02 0 0.86 n 379 0.13 0.02 0 1 n 523
within 0.49 0.24 -0.31 1.29 T-bar 66.1 0.49 0.24 -0.36 1.35 T-bar 61.9 0.49 0.24 -0.39 1.43 T-bar 37.3
detnia
overall 0.78 0.42 0.18 0 1 N 21,282 0.75 0.44 0.19 0 1 N 22,020 0.71 0.46 0.21 0 1 N 17,451
between 0.15 0.02 0.30 1 n 342 0.18 0.03 0 1 n 379 0.21 0.04 0 1 n 518
within 0.39 0.15 -0.22 1.47 T-bar 62.2 0.40 0.16 -0.24 1.45 T-bar 58.1 0.41 0.17 -0.28 1.54 T-bar 33.7
Dded
overall 0.94 0.24 0.06 0 1 N 21,262 0.88 0.32 0.10 0 1 N 22,017 0.88 0.33 0.11 0 1 N 17,462
between 0.06 0.00 0.29 1 n 342 0.09 0.01 0.5 1 n 379 0.09 0.01 0.5 1 n 518
within 0.23 0.05 -0.05 1.65 T-bar 62.2 0.32 0.10 -0.10 1.38 T-bar 58.1 0.32 0.10 -0.11 1.38 T-bar 33.7
Constructo: P - Effect pairs
dfilhos
overall 0.27 0.45 0.20 0 1 N 21,259 0.29 0.46 0.21 0 1 N 21,972 0.31 0.46 0.21 0 1 N 17,438
between 0.14 0.02 0 0.78 n 342 0.15 0.02 0 1 n 379 0.16 0.03 0 1 n 518
within 0.43 0.18 -0.51 1.22 T-bar 62.2 0.44 0.19 -0.53 1.25 T-bar 58 0.44 0.19 -0.61 1.27 T-bar 33.7
dirmão
overall 0.95 0.23 0.05 0 1 N 21,221 0.94 0.21 0.04 0 1 N 21,975 0.96 0.20 0.04 0 1 N 17,434
between 0.05 0.00 0.67 1 n 342 0.05 0.00 0.67 1 n 379 0.04 0.00 0.79 1 n 518
within 0.22 0.05 -0.04 1.28 T-bar 62.1 0.21 0.04 -0.04 1.29 T-bar 58 0.20 0.04 -0.04 1.27 T-bar 33.7
decivil
overall 0.62 0.49 0.24 0 1 N 21,121 0.62 0.49 0.24 0 1 N 21,970 0.62 0.49 0.24 0 1 N 17,442
between 0.14 0.02 0 1 n 342 0.15 0.02 0 1 n 379 0.15 0.02 0 1 n 518
within 0.47 0.22 -0.27 1.42 T-bar 61.8 0.47 0.22 -0.30 1.42 T-bar 58 0.47 0.22 -0.34 1.55 T-bar 33.7
Construct: F - Family and socioeconomic factors
despai
overall 0.15 0.35 0.12 0 1 N 21,204 0.15 0.35 0.12 0 1 N 21,943 0.12 0.33 0.11 0 1 N 17,371
between 0.10 0.01 0 0.59 n 342 0.10 0.01 0 0.67 n 379 0.10 0.01 0 0.51 n 518
Within 0.34 0.12 -0.44 1.13 T-bar 62 0.33 0.11 -0.52 1.13 T-bar 58 0.31 0.10 -0.38 1.11 T-bar 33.5
desmãe
Overall 0.12 0.32 0.10 0 1 N 21,167 0.13 0.33 0.11 0 1 N 22,001 0.12 0.32 0.10 0 1 N 17,433
Between 0.08 0.01 0 0.53 n 342 0.09 0.01 0 0.65 n 379 0.09 0.01 0 0.53 n 518
Within 0.31 0.10 -0.41 1.10 T-bar 62.2 0.32 0.10 -0.52 1.12 T-bar 58.1 0.31 0.10 -0.41 1.10 T-bar 33.7
Variables 2002 2003 2006
Mean SD Variance Min. Max. Observations Mean SD Variance Min. Max. Observations Mean SD Variance Min. Max. Observations
Construct: F – Family and socioeconomic factors
enmdpúb
overall 0.64 0.48 0.23 0 1 N 21,167 0.64 0.48 0.23 0 1 N 22,001 0.70 0.46 0.21 0 1 N 17,466
between 0.20 0.04 0.02 1 n 342 0.20 0.04 0 1 n 379 0.20 0.04 0 1 n 518
within 0.44 0.19 -0.34 1.62 T-bar 62.2 0.44 0.19 -0.35 1.51 T-bar 58.1 0.42 0.18 -0.27 1.55 T-bar 33.7
renda_1
overall 0.09 0.28 0.08 0 1 N 21,304 0.12 0.33 0.11 0 1 N 21,983 0.21 0.41 0.17 0 1 N 17,399
between 0.09 0.01 0 0.5 n 342 0.11 0.01 0 0.54 n 379 0.14 0.02 0 0.89 n 518
within 0.28 0.08 -0.41 1.08 T-bar 62.3 0.32 0.10 -0.42 1.11 T-bar 58 0.39 0.15 -0.68 1.19 T-bar 33.6
renda_2
overall 0.51 0.50 0.25 0 1 N 21,251 0.53 0.50 0.25 0 1 N 21,983 0.26 0.44 0.19 0 1 N 17,399
between 0.13 0.02 0.24 0.80 n 342 0.13 0.02 0 1 n 379 0.11 0.01 0 1 n 518
within 0.49 0.24 -0.29 1.37 T-bar 62.1 0.49 0.24 -0.31 1.35 T-bar 58 0.43 0.18 -0.40 1.22 T-bar 33.6
renda_3
overall 0.28 0.45 0.20 0 1 N 21,252 0.23 0.42 0.18 0 1 N 21,983 0.33 0.47 0.22 0 1 N 17,399
between 0.11 0.01 0 0.67 n 342 0.12 0.01 0 1 n 379 0.12 0.01 0 1 n 518
within 0.44 0.19 -0.39 1.25 T-bar 62.1 0.41 0.17 -0.30 1.21 T-bar 58 0.46 0.21 -0.31 1.29 T-bar 33.6
renda_4
overall 0.11 0.31 0.10 0 1 N 21,251 0.07 0.26 0.07 0 1 N 21,983 0.12 0.32 0.10 0 1 N 17,399
between 0.09 0.01 0 0.50 n 342 0.06 0.00 0 1 n 379 0.08 0.01 0 0.41 n 518
within 0.30 0.09 -0.39 1.10 T-bar 62.1 0.25 0.06 -0.43 1.06 T-bar 58 0.31 0.10 -0.29 1.10 T-bar 33.6
renda_5
overall 0.02 0.12 0.01 0 1 N 21,251 0.04 0.19 0.04 0 1 N 21,983 0.05 0.21 0.04 0 1 N 17,399
between 0.03 0.00 0 0.18 n 342 0.04 0.00 0 0.29 n 379 0.05 0.00 0 0.24 n 518
within 0.12 0.01 -0.16 1.01 T-bar 62.1 0.18 0.03 -0.26 1.03 T-bar 58 0.20 0.04 -0.20 1.03 T-bar 33.6
renda_6
overall 0.02 0.15 0.02 0 1 N 17,399
between 0.03 0.00 0 0.16 n 518
within 0.15 0.02 -0.14 1.02 T-bar 33.6
renda_7
overall 0.02 0.13 0.02 0 1 N 17,399
between 0.03 0.00 0 0.2 n 518
within 0.13 0.02 -0.18 1.01 T-bar 33.6
Construct: R – Resources of courses and institutions (educational process)
dexten
overall 0.11 0.31 0.10 0 1 N 21,204 0.11 0.31 0.10 0 1 N 21,993 0.13 0.33 0.11 0 1 N 17,407
between 0.12 0.01 0 0.84 n 342 0.12 0.01 0 1 n 379 0.13 0.02 0 1 n 518
within 0.30 0.09 -0.73 1.10 T-bar 62 0.30 0.09 -0.59 1.10 T-bar 58 0.31 0.10 -0.74 1.11 T-bar 33.6
Variables 2002 2003 2006
Mean SD Variance Min. Max. Observations Mean SD Variance Min. Max. Observations Mean SD Variance Min. Max. Observations
Construct: R – Resources of courses and institutions (educational process)
Dic
Overall 0.06 0.23 0.05 0 1 N 21,204 0.25 0.43 0.18 0 1 N 21,993 0.27 0.44 0.19 0 1 N 17,407
Between 0.06 0.00 0 0.56 n 342 0.16 0.03 0 1 n 379 0.16 0.03 0 1 n 518
Within 0.23 0.05 -0.50 1.05 T-bar 62 0.41 0.17 -0.63 1.24 T-bar 58 0.42 0.18 -0.57 1.25 T-bar 33.6
SOURCE: Micro data of the Provão 2002/2003 and Enade/2006.
Level 1 independent variables: nidade = Current age informed; dsex = 1 female, 0 masculine; detnia = 1 whites and yellows, 0 blacks and browns; dded = 1 spent at least one hour studying
beyond classroom, 0 contrary case; dfilhos = 1 has son/daughter (s), 0 otherwise; dirmão = 1 if has sibling (s), 0 otherwise; decivil = 1 unmarried, 0 married, separated, widowed and others;
despai1 father with higher education, 0 otherwise; desmãe = 1 mother with higher education, 0 otherwise; enmdpúb = 1 attended the high school in the public school, 0 otherwise. income_1 = 1
family income range up to R $ 720,00, and 0 otherwise; income _2 = 1 family income range from R $ 721,00 to R $ 2.400,00, 0 otherwise; income_3 = 1 family income range from R $ 2.401,00
to R $ 4.800,00, 0 otherwise; income_4 = 1 family income range from R $ 4.801,00 to R $ 7.200,00, 0 otherwise; income 5 = 1 family income range more than R $ 7.201,00. Values of the level 1
independent variables Enade 2006: 0 otherwise. income_1 = 1 family income range up to three minimum wages, and 0 otherwise; income _2 = 1 family income range more than 3 to 5 minimum
wages; 0 otherwise; income_3 = 1 family income range more than 5 up to 10 minimum wages, 0 otherwise; income 4 = 1 family income range over 10 to 15 minimum wages, 0 otherwise;
income_5 = 1 family income range over 15 to 20 minimum wages, 0 otherwise; income_6 = 1 family income range more than 20 to 30 minimum wages, 0 otherwise; income _7 = 1 family
income range more than 30 minimum wages; dexten = 1 participated in extension activities, 0 otherwise; dic = 1 participated at least in one of scientific initiation or technological, monitory, and
scientific project, 0 otherwise.
Tabela A2
Summary statistics of the exogenous variables of Provão 2002/2003 and Enade/2006 of Accountancy of Level 2 - Institutions
Variables 2002 2003 2006
Mean SD Variance Min. Max. Observations Mean SD Variance Min. Max. Observations Mean SD Variance Min. Max. Observations
Construct: R – Resources of courses and institutions (Teachers)
md
overall 0.47 0.23 0.05 0 1 N 20,334 0.51 0.19 0.04 0 0.99 N 23,182 0.54 0.19 0.04 0 0.95 N 18,614
between 0.22 0.05 0 1 n 313 0.19 0.04 0 0.99 n 369 0.19 0.04 0 0.95 n 488
within 0 0.47 0.47 T-bar 65 0 0.51 0.51 T-bar 62.8 0 0.54 0.54 T-bar 38.1
v_intg
overall 0.20 0.26 0.07 0 1 N 18,840 0.30 0.29 0.08 0 1 N 21,577 0.30 0.28 0.08 0 1 N 16,749
between 0.26 0.07 0 1 n 287 0.28 0.08 0 1 n 344 0.25 0.06 0 1 n 428
within 0 0.20 0.20 T-bar 65.6 0 0.30 0.30 T-bar 62.8 0 0.30 0.30 T-bar 39.1
dmcont1
overall 0.87 0.10 0.01 0.44 1 N 22,643 0.90 0.09 0.01 0.33 1 N 23,451 0.92 0.08 0.01 0.39 1 N 19,339
between 0.11 0.01 0.44 1 n 342 0.10 0.01 0.33 1 n 379 0.09 0.01 0.39 1 n 518
within 0 0.87 0.87 T-bar 66.2 0 0.90 0.90 T-bar 61.9 0 0.92 0.92 T-bar 37.3
42
Variables 2002 2003 2006
Mean SD Variance Min. Max. Observations Mean SD Variance Min. Max. Observations Mean SD Variance Min. Max. Observations
Construct: I – Characteristics of the educational system
dorgac
Overall 0.17 0.37 0.14 0 1 N 22,659 0.62 0.49 0.24 0 1 N 23,401 0.55 0.50 0.25 0 1 N 19,497
between 0.37 0.14 0 1 n 343 0.50 0.25 0 1 n 377 0.49 0.24 0 1 n 523
within 0 0.17 0.17 T-bar 66.1 0 0.62 0.62 T-bar 62.1 0 0.55 0.55 T-bar 37.3
dcatad
overall 0.86 0.34 0.12 0 1 N 22,659 0.76 0.43 0.18 0 1 N 23,401 0.80 0.40 0.16 0 1 N 19,497
between 0.34 0.12 0 1 n 343 0.39 0.15 0 1 n 377 0.35 0.12 0 1 n 523
within 0 0.86 0.86 T-bar 66.1 0 0.76 0.76 T-bar 61.9 0 0.80 0.80 T-bar 37.3
Construct: R – Resources of courses and institutions (educational process)
estapr1
overall 0.92 0.05 0.67 1 N 22,643 0.90 0.06 0.5 1 N 23,451 0.84 0.08 0.5 1 N 19,339
between 0.06 0.00 0.67 1 n 342 0.08 0.00 0.5 1 n 379 0.09 0.01 0.5 1 n 518
within 0 0.00 0.92 0.92 T-bar 66.2 0 0.01 0.90 0.90 T-bar 61.9 0 0.01 0.84 0.84 T-bar 37.3
mat1
overall 0.93 0.07 0.33 1 N 22,643 0.94 0.06 0.5 1 N 23,451 0.95 0.06 0.67 1 N 19,339
between 0.08 0.00 0.33 1 n 342 0.07 0.00 0.5 1 n 379 0.06 0.00 0.67 1 n 518
within 0 0.01 0.93 093 T-bar 66.2 0 0.00 0.94 0.94 T-bar 61.9 0 0.00 0.95 0.95 T-bar 37.3
aulexp1
overall 0.27 0.13 0 0.65 N 22,643 0.23 0.11 0 1 N 23,451 0.002 0.001 0 0.006 N 19,339
between 0.13 0.02 0 0.65 n 342 0.13 0.01 0 1 n 379 0.001 0.00 0 0.006 n 518
within 0 0.02 0.27 0.27 T-bar 66.2 0 0.02 0.23 023 T-bar 61.9 0 0.00 0.002 0.002 T-bar 37.3
csi1
overall 0.86 0.12 0.13 1 N 22,643 0.88 0.11 0.41 1 N 23,451
between 0.14 0.01 0.13 1 n 342 0.12 0.01 0.41 1 n 379
within 0 0.02 0.86 0.86 T-bar 66.2 0 0.01 0.88 0.88 T-bar 61.9
pltr1
overall 0.87 0.10 0.21 1 N 22,643 0.89 0.09 0.49 1 N 23,451
between 0.11 0.01 0.21 1 n 342 0.10 0.01 0.49 1 n 379
within 0 0.01 0.87 0.87 T-bar 66.2 0 0.01 0.89 0.89 T-bar 61.9
Construct: P – Effects of the pairs
esmãe1
overall 0.12 0.08 0.01 0 0.53 N 22,643 0.13 0.09 0.01 0 0.65 N 23,451 0.12 0.09 0.01 0 0.53 N 19,339
between 0.08 0.01 0 0.53 n 342 0.09 0.01 0 0.65 n 379 0.09 0.01 0 0.53 n 518
within 0 0.12 0.12 T-bar 66.2 0 0.13 0.13 T-bar 61.9 0 0.12 0.12 T-bar 37.3
43
Variables 2002 2003 2006
Mean SD Variance Min. Max. Observations Mean SD Variance Min. Max. Observations Mean SD Variance Min. Max. Observations
Construct: P – Effects of the pairs
espai1
overall 0.15 0.10 0.01 0 0.59 N 22,643 0.15 0.10 0.01 0 0.67 N 23,451 0.12 0.10 0.01 0 0.51 N 19,339
between 0.10 0.01 0 0.59 n 342 0.10 0.01 0 0.67 n 379 0.10 0.01 0 0.51 n 518
within 0 0.15 0.15 T-bar 66.2 0 0.15 0.15 T-bar 61.9 0 0.12 0.12 T-bar 37.3
midade
overall 27.88 2.03 4.12 23 39.89 N 22,615 28.68 2.07 4.28 23.13 42.5 N 23,451 29.04 2.56 6.55 22.96 48 N 19,339
between 2.57 6.60 23 39.89 n 342 2.64 6.97 23.13 42.5 n 379 2.94 8.64 22.96 48 n 518
within 0 27.88 27.88 T-bar 66.1 0 28.68 28.68 T-bar 61.9 0 29.04 29.04 T-bar 37.3
SOURCE: Micro data of the Provão 2002/2003 and Enade/2006
Level 2 independent variables: md = Proportion of teachers with master and doctor degree; v_intg = Proportion of teachers with full-time working; dmcont1 = Proportion of responses given by
students regarding IES to have teachers who demonstrated the mastery of content; dorgac = 1 Universities or University Centers, 0 for cases to be Integrated Colleges, Colleges, Schools, and
Institutes, Tecnological Education Centers; estapr1 = Ratio of responses given by students regarding IES to have teachers who conducted research activities as learning strategy; mat1 =
Proportion of responses given by students regarding IES to have teachers who indicated the use of books, copies of book chapters and handouts; aulexp1 = Proportion of responses given by
students regarding IES to have the majority of teachers whose teaching practice are predominantly lectures; csi1 = Proportion of responses given by students in relation to IES that provided the
knowledge on accounting information system; pltr1 = Proportion of responses given by students regarding IES that the issue ‘tax planning’ was considered in the course; espai1= Proportion of
fathers with higher education in IES; esmãe1 = Proportion of mothers with higher education in IES; midade = Students’ mean age in IES.