american evaluation association - IMPAQ International
19
IMPAQ INTERNATIONAL LLC. PRESENTERS: DR. JAIME MEZA-CORDERO DR. MICHAELA GULEMETOVA
-
Upload
khangminh22 -
Category
Documents
-
view
0 -
download
0
Transcript of american evaluation association - IMPAQ International
IMPAQ INTERNATIONAL LLC. PRESENTERS: DR. JAIME MEZA-CORDERO
DR. MICHAELA GULEMETOVA
EVALUATION OF THE AVANCEMOS PROGRAM
AMERICAN EVALUATION ASSOCIATION
AMERICAN EVALUATION ASSOCIATION
THE AVANCEMOS PROGRAM
§ In 2006, the secondary school attendance rate in Costa Rica reached an alarming low of 78.4%.
§ Local experts identified the increase in poverty rates as one of the main causes of school drop-outs.
§ The Government of Costa Rica (GCR) developed in 2007 a national-wide Conditional Cash Transfer program (CCT) to combat this problematic.
AMERICAN EVALUATION ASSOCIATION
SELECTION CRITERIA
§ The family needs to be verified by government institutions as in poverty of extreme poverty.
§ The family needs to apply for the inclusion of their adolescents into the program.
§ The government provides a monthly transfer conditional on school attendance and completion.
AMERICAN EVALUATION ASSOCIATION
EVALUATION OBJECTIVE
§ UNICEF provided funding to the GCR to evaluate the intervention and identify mid-course corrections.
§ IMPAQ conducted an Implementation and Impact Evaluation for UNICEF and GCR. The project included an Action Plan to develop the evaluation recommendations.
AMERICAN EVALUATION ASSOCIATION
RESEARCH QUESTIONS
§ How is the Avancemos program performing?
§ What have been the effects of the program on:
§ School Attendance and School Completion?
§ Hours Worked?
§ Gender, Regional, and Age group differentials?
AMERICAN EVALUATION ASSOCIATION
EVALUATION DESIGN
§ Mixed methods design; quantitative analysis and qualitative component to help explain the mechanisms that generate effects.
§ Program implementation aspects were studied through key informant interviews and focus groups.
§ Program impacts were measured through Propensity Score Matching methods using Secondary Data from the National Household Survey.
AMERICAN EVALUATION ASSOCIATION
EMPIRICAL STRATEGY
AVANCEMOS NO AVANCEMOS
CONTROLTREATMENT
PROPENSITY SCORE MATCHING
AMERICAN EVALUATION ASSOCIATION
DATA § Key Informant Interviews were conducted with local experts on
education and poverty and with government officials.
§ 44 focus groups across were conducted across a representative sample of discricts of the entire country with beneficiaries and their mothers.
§ Quantitative data comes from the National Household Survey (ENAHO) prepared by the Census Bureau of Costa Rica (INEC). The data used for this project includes 2006 through 2014 yearly datasets.
AMERICAN EVALUATION ASSOCIATION
DESCRIPTIVE STATISTICS
AMERICAN EVALUATION ASSOCIATION
DESCRIPTIVE STATISTICS
2006 2007 2008 2009 2010 2011 2012 2013 2014PercentageofWorkingAdolescents(doesnotincludedomesticwork)
Non-Avancemos 25 27 26 22 22 22 20 19 18
Avancemos 7 3 5 5 5 5 4 3Total 25 26 23 19 18 19 17 16 15
AverageHoursWorkedPercentageofWorkingAdolescents(doesnotincludedomesticwork)
Non-Avancemos 41 43 44 42 44 39 52 36 48
Avancemos 24 23 27 27 21 0 20 0Total 41 42 44 41 44 39 52 32 48
AMERICAN EVALUATION ASSOCIATION
SCHOOL DROP-OUT REASONS (AGE 12-17)
0%
5%
10%
15%
20%
25%
30%
35%
40%
2006 2007 2008 2009 2010 2011 2012 2013 2014
Not interestedSchool costsWork for wagesOtherAcademic difficultiesNo school accessMarriage/pregnancyIllness/disabilityHousework
AMERICAN EVALUATION ASSOCIATION
PSM FINDINGSSchool
AttendanceYears of
SchoolingWork Hours
Workedn
All 0.2672* 0.8944* -0.0805* -0.0880* 6799Male 0.2428* 0.7664* -0.0292* -0.0689 3295
Female 0.2929* 1.0109* -0.1368* -0.1125 3504Urban 0.1791* 0.5995* -0.0656* -0.1172* 3081Rural 0.3116* 1.0202* -0.0867* -0.0714 3718
Ages 12-14 0.0635* 1.1170* -0.0052* n/a 1903Ages 15-17 0.2851* 0.9376* -0.0468* -0.1432 2158Ages 18-21 0.5434* 0.4518* -0.2109* -0.1604* 2738
AMERICAN EVALUATION ASSOCIATION
PROGRAM STRENGHTS
§ National coverage and high administrative capacity of implmementing agency for over 9 years.
§ Timely payment of transfer
§ Positive perception from beneficiaries: students and parents.
AMERICAN EVALUATION ASSOCIATION
PROGRAM CHALLENGES§There is no timely communication of program changes and requisites to reach the most vulnerable population.
§The transition from the primary school scholarship basedsystem to the secondary school CCT is not smooth.
§The transfer amount lacks a technical criteria to account forchanges in the cost of living.
AMERICAN EVALUATION ASSOCIATION
RECOMMENDATIONS § Unify and communicate the program application requirements
across all regional offices of the implementing agency.
§ Develop strategies to reach the most vulnerable population.
§ Improve the program monitoring system and collaborate closelywith the Ministry of Education to receive timely verification of the school attendance conditionality.
§ Add a component of school quality into the program and updatethe transfer amount periodically based on a technical criteria.
AMERICAN EVALUATION ASSOCIATION
CONTACT INFORMATION
Dr. Jaime Meza-Cordero510.597.2419
Dr. Michaela Gulemetova202.774.1956
