World Bank Document - Clearinghouse for Financing ...

Document of

The World Bank

FOR OFFICIAL USE ONLY

Report No: ICR00004986

IMPLEMENTATION COMPLETION AND RESULTS REPORT

TF-17115 and TF-A5617

ON A

GRANT

IN THE AMOUNT OF US$15.5 MILLION

TO THE

FEDERAL DEMOCRATIC REPUBLIC OF ETHIOPIA

FOR THE

ETHIOPIA STATISTICS FOR RESULTS PROJECT

MARCH 25, 2020

Poverty and Equity Global Practice Africa Region

This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS Currency Unit

1 USD = Ethiopian Birr (ETB) 32.60 ETB (as of March 23, 2020)

FISCAL YEAR July 8 – July 7

ABBREVIATIONS AND ACRONYMS

AF Additional Financing AFDB African Development Bank CAPI Computer-Assisted Personal Interviewing CPF Country Partnership Framework CPS Country Partnership Strategy CSA Central Statistical Authority EDQAF Ethiopian Data Quality Assessment Framework ESMF Environmental and Social Management Framework ESPES Enhancing Shared Prosperity through Equitable Services FAO Food and Agriculture Organization GIS Geographic Information System GoE Government of Ethiopia GPS Global Positioning System GTP Growth and Transformation Plan HQ Headquarters IP Implementation Progress ICR Implementation Completion and Results Report ICT Information and Communication Technology ISM Implementation Support Mission ISR Implementation Status and Results Report IT Information Technology LSMS M&E

Living Standards Measurement Study Monitoring and Evaluation

MDAs Ministries, Departments, and Agencies MDG Millennium Development Goal MOF Ministry of Finance MTR Midterm Review NSDS National Statistical Development Strategy NSS National Statistical System PAD Project Appraisal Document PDO Project Development Objective PIU Project Implementation Unit RF Results Framework SCI Statistical Capacity Index SFR Statistics for Results SRF-CF Statistics for Results Facility Catalytic Fund STEP Systematic Tracking of Exchanges in Procurement USAID United States Agency for International Development

Regional Vice President: Hafez M. H. Ghanem Country Director: Carolyn Turk Regional Director: Asad Alam Practice Manager: Pierella Paci Task Team Leaders: Tom Bundervoet, Waleed Malik, Senait Yifru ICR Main Contributors: Berhe Beyene, Senait Yifru, and Melat Asfaw

TABLE OF CONTENTS

DATA SHEET .......................................................................................................................... 1

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5

A. CONTEXT AT APPRAISAL ......................................................................................................5

II. OUTCOME ..................................................................................................................... 15

A. RELEVANCE OF PDOs ............................................................................................................ 15

B. ACHIEVEMENT OF PDOs (EFFICACY) ................................................................................... 16

C. EFFICIENCY ........................................................................................................................... 17

D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 19

E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 19

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 20

A. KEY FACTORS DURING PREPARATION ................................................................................... 20

B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 21

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 22

A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 22

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 23

C. BANK PERFORMANCE ........................................................................................................... 24

D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 25

V. LESSONS AND RECOMMENDATIONS ............................................................................. 26

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 28

ANNEX 2: KEY OUTPUTS BY COMPONENT ............................................................................ 37

ANNEX 3. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 40

ANNEX 4. PROJECT COST BY COMPONENT ........................................................................... 42

ANNEX 5. LIST OF TRAINING BY TOPICS, LOCATION, AND NUMBER OF TRAINEES BY YEAR .... 43

The World Bank Ethiopia Statistics for Results Project (P147356)

Page 1 of 45

DATA SHEET

BASIC INFORMATION Product Information Project ID Project Name

P147356 Ethiopia Statistics for Results Project

Country Financing Instrument

Ethiopia Investment Project Financing

Original EA Category Revised EA Category

Partial Assessment (B) Partial Assessment (B)

Organizations

Borrower Implementing Agency

Federal Democratic Republic of Ethiopia, Federal Democratic Republic of Ethiopia

Central Statistics Agency

Project Development Objective (PDO)

Original PDO The objective of the Project is to enhance the capacity of the Central Statistical Authority (CSA) at the organizational, human, and physical levels in order to produce and disseminate reliable, accessible, and timely statistics.


Page 2 of 45

FINANCING

Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing TF-17115 10,000,000 10,000,000 10,000,000

TF-A5617 5,500,000 5,428,228 5,428,228

Total 15,500,000 15,428,228 15,428,228

Non-World Bank Financing 0 0 0

Borrower/Recipient 0 0 0

Total 0 0 0

Total Project Cost 15,500,000 15,428,228 15,428,228

KEY DATES

Approval Effectiveness MTR Review Original Closing Actual Closing 30-May-2014 13-Jun-2014 12-Jun-2017 30-Jun-2019 30-Sep-2019

RESTRUCTURING AND/OR ADDITIONAL FINANCING

Date(s) Amount Disbursed (US$M) Key Revisions 15-Jun-2017 6.65 Change in Loan Closing Date(s)

Change in Implementation Schedule 20-Feb-2018 9.83 Additional Financing

Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Financing Plan Change in Disbursements Arrangements Change in Procurement Change in Implementation Schedule

19-Apr-2019 13.90 Change in Loan Closing Date(s)


Page 3 of 45

KEY RATINGS

Outcome Bank Performance M&E Quality

Satisfactory Satisfactory Substantial

RATINGS OF PROJECT PERFORMANCE IN ISRs

No. Date ISR Archived DO Rating IP Rating Actual

Disbursements (US$M)

01 10-Nov-2014 Moderately Satisfactory Moderately Satisfactory 1.00

02 29-Jun-2015 Moderately Satisfactory Moderately Satisfactory 1.22

03 10-Feb-2016 Moderately Satisfactory Satisfactory 3.34

04 31-Oct-2016 Moderately Satisfactory Moderately Satisfactory 4.36

05 21-Jun-2017 Satisfactory Satisfactory 7.36

06 27-Dec-2017 Satisfactory Satisfactory 9.83


08 26-Dec-2018 Moderately Satisfactory Moderately Satisfactory 12.55


SECTORS AND THEMES

Sectors Major Sector/Sector (%)

Public Administration 100

Other Public Administration 100 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%)


Page 4 of 45

Economic Policy 9 Economic Growth and Planning 9

Inclusive Growth 3

Structural Transformation and Economic Diversification 3

Green Growth 3

Private Sector Development 50

Enterprise Development 50 MSME Development 50

Public Sector Management 40

Public Administration 20 Administrative and Civil Service Reform 20

Data Development and Capacity Building 20

Data production, accessibility and use 20

ADM STAFF

Role At Approval At ICR

Regional Vice President: Makhtar Diop Hafez M. H. Ghanem

Country Director: Guangzhe Chen Carolyn Turk

Director: Marcelo Giugale Carolina Sanchez

Practice Manager: Pablo Fajnzylber Pierella Paci

Task Team Leader(s): Waleed Haider Malik, Senait Kassa Yifru, Michael Tobias Geiger

Tom Bundervoet, Senait Kassa Yifru

ICR Contributing Author: Berhe Mekonnen Beyene


Page 5 of 45

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES

A. CONTEXT AT APPRAISAL

1. During project appraisal, the Government of Ethiopia (GoE) was implementing an ambitious Growth and Transformation Plan (GTP) 2010/11–2014/15, which set a long-term goal of becoming a middle-income country by 2023, with growth rate of at least 11.2 percent per year during the plan period. To achieve the GTP goals and objectives, the GoE had followed a ‘developmental state’ model with a strong role for the Government in many aspects of the economy. It had prioritized key sectors, such as industry and agriculture, as drivers of sustained economic growth and job creation. The GTP identified sectoral targets and strategies to enhance economic growth and achieve medium- and long-term objectives. The development strategy had also highlighted the importance of data to track and monitor achievements of targets underlined in the plan.

2. A stronger and more functional National Statistical System (NSS) is a key instrument to monitor and improve performance toward the objectives set by the second National Statistical Development Strategy (NSDS II), which covers 2015/16 to 2019/20. Ethiopia’s NSDS I was established in 2008/09 and originally covers the implementation period 2009/10–2013/14 but was amended by one more year to align with GTP to 2010/11–2014/15. The NSDS incorporated six strategic themes to strengthen the overall NSS, which is led by the Central Statistical Authority (CSA). Based on the NSDS themes, the SFR program concept identified three main categories of challenges:

(a) Low institutional and infrastructural capacity in the CSA and statistics-producing government ministries, departments, and agencies (MDAs). The CSA lacked working space and modern data collection equipment which caused inefficiency in the agency’s operations and difficulty of staff to adapt to modernized data collection systems. Additionally, branch offices were dilapidated and lacked the necessary facilities to conduct their day-to-day duties.

(b) Data gaps. (c) Coordination problems within the NSS and in its interactions with stakeholders and partners.

3. As a coordinator of the NSS, the CSA is required to set the standards for the collection and compilation of statistics to ensure quality and reliability and to provide guidance to other producers and users of statistics.

4. At project approval in 2014, the Ethiopia Statistical Capacity Index (SCI) was 61.11. The SCI consistently improved after the project became effective in 2014 and relatively higher scores in 2018 (SCI 72.22). The increased SCI partly attributes to the CSA’s and NSS’s improved data production and the capacity-building activities supported by the project.

Rationale for World Bank Involvement

5. At appraisal, the rationale for the World Bank’s involvement was (a) World Bank comparative advantage in the strengthening of statistical capacity, (b) convening power, and (c) consistency with national and international commitments. The World Bank had engaged with the GoE to focus on the


Page 6 of 45

challenges facing in the NSS. These constraints included declining funding resources, low overall statistical capacity, and limited statistical and physical infrastructure.

6. The engagement of the World Bank in funding this project was critical because CSA was not able to secure sufficient resources from the Government’s general budget to enhance its institutional and infrastructural capacity. In addition, the World Bank also brought global knowledge in the area of data quality assurance and adherence to international standards. For example, since 2011, the Living Standards Measurement Study (LSMS) team of the World Bank have been collaborating with the CSA to produce the biennial Ethiopian Socio-Economic Survey.

Contribution to Higher-level Objectives

7. The project was designed to contribute to the goals of the GoE’s GTP and the Millennium Development Goals (MDGs) and facilitate the post-2015 MDG agenda setting. Improvements in the quality, relevance, and timeliness of priority statistics contribute to more effective development policy by (a) providing the basis for monitoring and evaluating the Government’s GTP and for tracking the progress of MDGs and (b) providing feedback to policy makers and citizens on the effectiveness of public policy and the use of public resources. In addition, the project was to contribute the goals outlined in the 2013–2016 Country Partnership Strategy (CPS) of Ethiopia, (Report no. 71884-ET). It was designed to support the NSDS, which was expected to provide the relevant indicators to be used in formulating, updating, monitoring, and evaluating the strategies and targets of the country’s social and economic development programs including its first and second GTPs.

Design and Implementation

8. The World Bank’s support to statistical capacity building in Ethiopia followed the principles of the Statistics for Results Facility Catalytic Fund (SRF-CF). The SRF-CF seeks to guide the support programs through a National Partnership Group and to base statistical capacity building on the country-level priorities identified in the NSDS. In fact, the SRF-CF aimed to close some of the NSDS implementation funding gaps, address capacity gaps highlighted by the document, and leverage additional financing (AF) through partners to meet the CSA’s and NSS’s long-term sustainability needs. The project was financed by Statistics for Results Facility-Catalytic Fund (SRF-CF) of US$10 million and an Additional Financing from SRF-CF of US$5.5 million.

9. The design and implementation of the SFR Project was expected to strengthen the capacity of the NSS including the CSA to improve the data needed for an evidence-based planning and decision making in the country. As mentioned above, the SFR’s Project Development Objective (PDO) was related to the goals outlined in the 2013–2016 CPS of Ethiopia. The project was also designed to support the NSDS, which is expected to provide the relevant indicators to be used in formulating, updating, monitoring, and evaluating the strategies and targets of the country’s social and economic development programs.

Theory of Change (Results Chain)

10. The Theory of Change presented in Figure 1 below presents project activities, outputs, and outcomes, and how these relate to the Project Development Objective (PDO) as stated in the Project Appraisal Document (PAD) and Financing Agreements. Activities under project management had an overall


Page 7 of 45

impact on all outcomes/objectives and contribute to the attainment of the PDO. In the long-term, the project supports the GoE in building the NSS that is capable and has capacity to provide policy makers with timely data and advice for evidence-based decision making and measure progress in achieving the Sustainable Development Goals (SDGs) and goals set under the NSDS II (2015/16 to 2019/20).

11. This long-term vision would not be attainable without investing in physical infrastructure, which is fundamental for collection, production, and dissemination of reliable and timely data. While physical infrastructure is crucial, timeliness and quality of the data cannot be achieved only by installing more hardware. It requires continuous capacity building of data users and producers, modernization of the CSA’s own internal structure, division of responsibilities between units within the CSA, and better institutional coordination and strengthened cooperation among the NSS’ key stakeholders, facilitating better use of data for design of better policies.

12. The assumption that the project contributed to the longer-term goal was well grounded in evidence. Besides accelerating progress toward higher levels of statistical capacity and a better governance structure, the project also increased the Government’s interest to develop the NSS which manifested itself in the request for a follow-on lending operation (currently under implementation). Implementing the new lending project will further pave the way for more outcomes, updated statistical infrastructure, and strengthened capacity of the CSA and MDAs staff.

13. The theory of change for the project builds on multiple critical assumptions which are essential to transform outputs into outcomes.


Page 8 of 45

Figure 1. Theory of Change

Note: EDQAF = Ethiopia’s Data Quality Assessment Framework; ESMF= Environmental and Social Management Framework; ICT = Information and communication technology.


Page 9 of 45

Project Development Objectives (PDOs)

14. The PDO is to enhance the capacity of the Central Statistical Authority (CSA) at the organizational, human, and physical levels in order to produce and disseminate reliable, accessible, and timely statistics. According to the PAD, the main beneficiaries of the project were (a) the CSA; (b) the Ministry of Finance (MOF) and selected MDAs, regional governments, and city administration authorities; (c) Parliament; (d) judicial branch authorities; (e) policy makers and planners in the public and private sectors, including civil society organizations; (f) researchers, associations, and academia; (g) international, regional, and global organizations and entities; and (h) members of the general public, who will also benefit from better and faster access to information.

Key Expected Outcomes and Outcome Indicators

15. The key expected outcomes as captured in the PDO statement are the increased production and dissemination of reliable, accessible and timely statistics relevant for evidence-based policy making and other uses. Based on the PDO, the project was designed to achieve three key outcomes:

• Outcome 1: Reliability and Quality. The CSA applies internationally accepted statistical techniques and ICT applications in the collection, compilation, and authenticity verification of data and validates data sources and statistical products in select MDAs.

• Outcome 2: Timeliness. Timeliness and increased frequency of publication for key statistics in line with national standards as agreed upon by the CSA and MDAs.

• Outcome 3: Access. Data readily available to users through the Open Data Portal and eCSA Program on Wheels.

Original Components at Appraisal

16. The project comprised four components at appraisal.

17. Component 1: Organizational and Infrastructure Capacity Development in the NSS (US$4.90 million at appraisal, US$7.80 million during AF, and US$11.90 million actual). The component aimed to support improvement in organizational performance, staff productivity and data accessibility by strengthening the capacity of the CSA and MDAs’ staff, and improvement of physical and information technology (IT) infrastructure of the CSA. This component provided the CSA with new services through the original project and Additional Financing (AF), including the construction and operationalization of six selected CSA branch offices (i.e., Ambo, Bahir Dar, Hawassa, and Mekelle funded by the original project and Adama and Dessie funded by the AF) out of 25 CSA branch offices. In addition, CSA headquarters (HQ) training rooms were renovated, mini data centers at the CSA HQ were established and made functional, and IT kiosks for the storage of statistical information were created. The CSA HQ library was renovated, a new conference hall at the CSA HQ was constructed, and the CSA HQ new office building was furnished. The project also financially supported 41 CSA staff to attend graduate degree programs in various local universities and provided different local and international trainings for the CSA and MDAs’ staff (see Annex 5 for details).

18. Component 2: Statistical Data Development and Management (US$1.5 million at appraisal, US$3.20 million during AF, and US$1.63 million actual). This component provided support to strengthen


Page 10 of 45

the quality of statistical data production, compilation, and validation, including preparatory activities and technical assistance to help undertake selected NSDS II priority surveys. This includes sampling frame development, training, workshops, purchase and piloting of modern tools, and other technical assistance and logistical vehicles to help undertake priority surveys. Both the original project and the AF supported to conduct Pilot population survey, two agriculture surveys, statistical business survey, welfare monitoring survey, and household consumption expenditure survey. The planned mining and construction price surveys had been removed during the AF because it was not prioritized by the CSA and resources were reallocated to scaling up business statistics survey.

19. Component 3: Statistical Methodology, Standards and Data Quality Assurance and Information Dissemination (US$1.90 million at appraisal, US$2.40 million during AF, and US$0.41 million actual). This component aimed to strengthen the capacity of the CSA’s training and data quality assessment activities, that is, the application of EDQAF for those engaged in the production of administrative statistics. This component supported statistical data dissemination activities to the public and all stakeholders by promoting statistical awareness. In addition, under this component, the CSA developed its website and also finalized its new branding and logo.

20. Component 4: Monitoring and Evaluation (US$1.70 million at appraisal, US$2.10 million during AF, and US$1.49 million actual). The component supported SFR Project management activities, including procurement, financial management, monitoring and evaluation (M&E), and ESMF implementation requirements.

21. The project cost was US$10 million at appraisal. An AF from SRF-CF of US$5.5 million was also approved and disbursed. At project closure, a total of US$15.4 million (99.5 percent) of project funds was disbursed and accounted for.

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION

Revised PDOs and Outcome Targets

22. The project underwent three restructurings to extend the project closing date, to revise the Results Framework and to reallocate funds between components due to AF in the amount of US$5.5 million from the SRF-CF to scale up project activities. None of these entailed changing the PDO or the project components.

Restructuring #1

23. The first level II restructuring approved on June 15, 2017, extended the closing date of the initial project by 20 months from June 30, 2017, to February 28, 2019, and revised the Results Framework. The PDO-level indicators were reduced from five to four, the PDO indicators target values were adjusted, and the target dates for the PDO and implementation progress (IP) indicators were adjusted to align with the new project closing date.


Page 11 of 45

24. Afterwards, the PDO-level results indicators were as follows:

• Data quality.

Indicator 1: Number of Key NSS members who have applied the EDQAF. The unit of measure was changed from percentage to number and end target value was changed to 5 instead of 70 percent.

Indicator 2: Percentage of users who are satisfied with official statistics methodologies as determined by the user satisfaction survey. End target value was reduced from 100 percent to 97 percent.

• Timeliness. Indicator 3: Percentage execution rate of annual statistical release calendar, target value was 75 percent. (no change).

• Data access. Indicator 4: Percentage of users who are satisfied with the accessibility of official statistics as determined by users’ satisfaction survey. End target value was increased from 85 percent to 94 percent.

Restructuring #2

25. The second level II restructuring with the AF approved on February 20, 2018, added US$5.5 million to the original project. The project’s closing date was extended by four months from February 28, 2019, to June 30, 2019. This restructuring and AF retained the same PDO, project components, and PDO-level indicators. However, the Results Framework was revised to monitor progress on the scaled-up and new activities added under the intermediate indictors and the end targets dates for both the PDO and IP indicators were adjusted to reflect the additional time for implementation and new project closing date.

Restructuring #3

26. The third level II restructuring was approved on April 19, 2019, and extended the project closing date from June 30, 2019, to September 30, 2019. The Results Framework end target date was changed to align with the project’s new closing date.


Page 12 of 45

Table 1. PDO and Intermediate Level Indicators Achievement Of Targets

Project Development Objective: To enhance the capacity of the Central Statistical Authority (CSA) at the organizational, human, and physical levels in order to produce and disseminate reliable, accessible, and timely statistics.

Indicator Target and Actual Values Status Comments

PDO-level Indicators 1. Number of key NSS members who have applied the EDQAF manual

Target: 5. Actual: 5 Target fully achieved (100 %)

The NSS member who applied the EDQAF manual are the Ministry of Agriculture, Education, Health, Road, and Water Resources.

2. Percentage of users who are satisfied with official statistics methodologies as determined by the user satisfaction survey

Target: 97%, Actual: 80% Target substantial achieved (82.5%)

The preparation work for the Population and Housing Census which has been postponed several times has forced CSA to cancel several planned activities and that is likely to have reduced the level of achievement for this indicator.

3. Percentage of execution rate of annual statistical release calendar

Target: 75%, Actual: 71% Target substantially achieved (94.7 %)

The preparatory activities under the then planned Population and Housing Census affected most of the CSA’s resources including the yearly activities that were planned under the annual statistical release calendar.

4. Percentage of users who are satisfied with the accessibility of official statistics as determined by the user satisfaction survey

Target: 94%, Actual: 77% Target substantially achieved (82%)

The preparation work for the Population and Housing Census which has been postponed several times has forced CSA to cancel several planned activities and that is likely to have reduced level of achievement for this indicator.

Intermediate Results Indicators

1. Number of new branch office buildings that are operational

Target: 6, Actual: 6 Target fully achieved (100%)

Six branch offices in Mekelle, Hawassa, Bahir Dar, Ambo, Dessie, and Adama are fully operational. They provide the CSA with new services, including training rooms, mini-data centers, and IT kiosks for the storage of statistical information; mini-libraries; and space to accommodate additional staff.

2.Conference hall and training center established and operational at the CSA HQ

Target: Yes, Actual: Yes Target fully achieved (100%)

A new conference center with the carrying capacity of 450 people, with all the amenities has been constructed and is currently operational; the CSA


Page 13 of 45

training center is also established, and its facilities include a training hall, four syndicate rooms, library, and offices. Both facilities are equipped with modern IT facilities.

3.Number of video conference centers established and operational

Target: 8, Actual:6 Target substantially achieved (75%)

Six video conference centers have been set up in the CSA. The four branch offices, Ambo, Hawassa, Bahir Dar, and Mekelle, are connected to the two video conference centers in the HQ. Due to lack of foreign exchange, the remaining planned two video conferencing systems for Adama and Dessie have not been set up.

4.Number of staff from CSA and MDAs trained annually, on areas of organizational management, IT, and Statistics

Target: 1,525, Actual: 1,456 Target substantially achieved (95%)

CSA and MDAs staff capacity developed. Detailed training activities are described in Annex 5.

5.Number of female staff from CSA and MDAs trained annually, on areas of organizational management, IT, and Statistics

Target: 534, Actual: 542

Target fully achieved (101%)

CSA and MDAs female staff capacity developed. Detailed training activities are described in Annex 5.

6.Statistical Business Register developed and operational

Target: Yes, Actual: Yes


The CSA business register system has been developed and the staff from the Statistical Business Register Directorate have taken trainings abroad regarding the system.

7.Number of surveys supported by the project

Target: 6, Actual: 6


Pilot population survey, two agriculture surveys, statistical business survey, welfare monitoring survey, and household consumption expenditure survey were supported by the project.

8.Number of people trained in the areas of statistical data production

Target: 387, Actual: 198 Target partially achieved (51%)

The budget allocated for this activity was shifted to other activities due to some restriction of international travel as per the GoE. Detailed training activities are described in Annex 5.

9.Number of CSA and MDAs staff trained in specialized quality assurance topics including EDQAF survey methods, poverty

Target: 2,944, Actual: 2,930


Target full achieved. Detailed training activities are described in Annex 5.


Page 14 of 45

mapping techniques, gender analysis of household surveys 10.Number of female CSA and MDAs staff trained in specialized quality assurance topics including EDQAF survey methods, poverty mapping techniques, gender analysis of household surveys

Target: 765, Actual:719

Target substantially achieved (94%)

This IP indicator was added during project restructuring in order to measure female staff training participants. Detailed training activities are described in Annex 5.

11.User-friendly CSA website established

Target: Yes, Actual: Yes Target fully achieved The CSA has developed a new website which is more user-friendly and easily accessible.

12.Number of user satisfaction survey reports

Target: 3, Actual: 3 Target fully achieved (100 %)

Three user satisfaction survey reports produced

13.Finalize NSDS II

Target: Yes, Actual: Yes Target fully achieved (100 %)

The project supported the preparation and implementation of NSDS II.

14.Number of audit reports produced

Target: 5, Actual: 5 Target fully achieved (100 %)

Audit reports produced timely

15.CSA project coordination team and PIU established and operational

Target: Yes, Actual: Yes Target fully achieved (100 %)

The CSA’s PIU was established after the project’s approval in 2014. Since then, the team has been fully operational until the project closing.

16.Number of project evaluation reports produced

Actual: 2, Target: 2 Target fully achieved (100%)

The project supported CSA’s to prepare midterm and end-term reviews.

17. Number of eCSA buses in use Actual: No, Target: No Cancelled The plan to implement an e-bus was canceled because it was too difficult to procure a specialized bus in the country and internationally. As a result, this activity was replaced by other dissemination activities including infographic competitions and statistical awareness activities on television, radio, and other media.


Page 15 of 45

Revised Components

27. The main components of the project did not change, but the activities, descriptions, and scope of various elements within the original components were revised during the AF in 2018. Component 2: Support of Statistical Data Development and Management, including development of mining, energy, construction, and transport sector statistics, was changed to supporting social, poverty, and price statistics in the AF.

Rationale for Changes and Their Implication on the Original Theory of Change

28. As described above, the restructurings of the project (Restructuring #1, Restructuring #2, and Restructuring #3) were necessary for efficient achievement of the remaining project activities, and to attain project outcomes and objectives.

II. OUTCOME

A. RELEVANCE OF PDOs

Assessment of Relevance of PDOs and Rating: The relevance of the Project Development Objective is rated High.

29. Even though the project underwent three restructurings, the PDO remained the same. The PDO is highly relevant to the World Bank’s Country Partnership Framework (CPF) (2018–2022, Report No. 119576-ET1). The objective of the CPF is to foster sustainable inclusive growth through job creation and economic transformation. The broad objective of the project, to enhance the capacity of the CSA at the organizational, human, and physical levels to produce and disseminate reliable, accessible, and timely statistics remains highly relevant. At completion, the PDO remained highly responsive and aligned to the country’s development priorities, as indicated in the GTP II2, which calls for improved data for policy decision making. This NSDS II, supported by the project, was widely disseminated and now serves as a source document that guides statistical reforms and statistical interventions.

30. The PDO was particularly relevant to and contributed to strengthen the national M&E and statistical system. It underpinned the World Bank’s efforts to work with the Government and with other development partners to support the GoE’s efforts to improve the capacity of the NSS to produce high-quality and timely statistics during the CPF period and beyond. The relevance of the PDO is also evident in the follow-on World Bank supported project - Enhancing Shared Prosperity through Equitable Services (ESPES) Project (P151432). The SFR and ESPES projects share similar activities and both support priorities outlined on the NSDS I and II. Both projects support CSA in data production and dissemination; ICT infrastructure, systems and tools for data production; and capacity building of CSA staff and NSS members. The SFR project laid the groundwork for these activities by improving the physical infrastructure in the CSA HQ and select branch offices and also increased the use of ICT in conducting surveys.

1 World Bank. 2017. Country Partnership Framework for the Federal Republic of Ethiopia for the Period FY18-FY22, Report No. 115135. 2 World Bank. 2016. Federal Democratic Republic of Ethiopia. Second Growth and Transition Plan.


Page 16 of 45

31. The ESPES Project, which is under implementation has two components: the first component supports data production and dissemination including supporting the welfare monitoring system, economic and price statistics, and improving data access, dissemination, and use. The second component supports organizational and statistical infrastructure and capacity building and improving the quality of statistics includes supporting the improvement of statistical, organizational, and physical infrastructure. Therefore, theses project activities are linked with the SFR Project activities and clearly show that the SFR Project remains relevant to the Government in particular to strengthen the national statistical system.

B. ACHIEVEMENT OF PDOs (EFFICACY)

32. The project substantially achieved its objectives (intended outcomes). Therefore, the efficacy rating is Substantial.

33. Over the course of the project, enhancing the capacity of the CSA at the organizations, human, and physical levels in order to produce and disseminate reliable, timely and accessible statistics for evidence-based policy making and other uses have increased significantly. Based on the PDO statement, there were three intended outcomes:

Outcome 1: Reliability and Quality. The CSA applies internationally accepted statistical techniques and ICT applications in the collection, compilation, and authenticity verification of data and validates data sources and statistical products in select MDAs.

Outcome 2: Timeliness. Timeliness and increased frequency of publication for key statistics in line with national standards as agreed upon by the CSA and MDAs.

Outcome 3: Access. Data readily available to users through the Open Data Portal and eCSA Program on Wheels.

34. The three key outcomes in the PDO reflect the longer-term effort by the NSS to provide timely and relevant data for evidence-based policy-making and better coordination among stakeholders. As demonstrated in the Theory of Change, the project intended to provide an integrated package of support to the NSS focusing on the CSA and selected MDAs to build capacity—both human capital and physical and statistical infrastructure. It envisioned that project support would contribute to a longer-term outcome of a strengthened NSS with an enhanced capacity to provide timely and reliable statistics relevant for evidence-based policy making. The three outcomes identified were substantially achieved. At project completion, out of the four PDO-level indicators, one is fully achieved, namely ‘number of key NSS members who have applied the EDQAF manual’, and the other three PDO-level indicators were substantially achieved (i.e., by more than 80 percent). Out of the 17 intermediate results indicators, 12 were fully achieved, while the remaining 3 were achieved above 95 percent, one IP indicator was achieved by 51 percent, and one IP indicator (Number of eCSA buses in use) was cancelled. Table 1 provides an assessment of the project achievement under each of the PDO level indicators and intermediate indicators.


Page 17 of 45

C. EFFICIENCY

Assessment of Efficiency and Rating 35. Based on the analysis provided below, efficiency rating is Substantial.

Economic Analysis

36. At the time of appraisal, economic benefits of the SFR Project were focused on the production and dissemination of better statistical data by improving the efficiency of CSA to produce higher-quality data for evidence-based decision making at policy, program, and project levels. As a result, the SFR Project started setting up the building blocks for a strong statistical institute that can produce and disseminate better quality statistics, hence contributing significantly to evidence-based policy making in Ethiopia.

Financial Analysis

37. The project was not amenable to financial analysis because national statistical agencies have very limited opportunities to recover the costs of data collection and analysis. National statistics are generally considered a public good and are usually financed from government revenue. Therefore, only marginal financial returns are expected from this project. However, the project outcomes were expected to promote improved allocation of resources by enhancing the monitoring of the impact of government spending by providing more accurate information on economic activities to the public at large The financing approach adopted is cost effective, given that the CSA coordinates its work with development partners on a regular basis.

38. The type of investments implemented by the CSA covered the essential aspect of its mandate which is data production and dissemination, as evidenced by the actual expenditure under each component. CSA had utilized 99.5 percent of the budget allotted for the project by the end of September 2019 (at project closing).

39. Investment in the infrastructure of the CSA. The working environment of the CSA and NSS staff has significantly improved because of the newly constructed branch offices in the six selected cities and the new conference center, renovated training center, and library in the CSA HQ. All these construction works were done at a reasonably low cost compared to the private sector in Ethiopia or similar projects in Africa. The newly introduced ICT technologies, including an upgraded data center (at the HQ) and video conferencing system (at the HQ and four new branch offices), have enabled staff to conduct local and international trainings easily from their premises and will reduce the cost of hotel expenses and other related costs. These office spaces have also increased the staff’s productivity and will benefit the CSA and NSS in the short and long term. Additionally, the project supported the procurement of 2,300 tablets for data collection activities, which improved the quality of the statistical data since manual data collection practice is now changed to a digital system. The application of tablets for data collection reduced cost of labor and costs of error correction and also eliminated the need for a time- and resource-consuming data-entry phase. Overall, the decision made by the borrower and the World Bank on prioritizing some of the major project activities accounted for the efficiency gains of the project.


Page 18 of 45

Aspects of Design and Implementation

40. The performance of the SFR Project on resource utilization and delivery of desired outcomes is generally rated as Satisfactory based on the overall management approach and the financial management undertaken during both the design and implementation of the project. Table 2 summarizes the budget utilization of the project. Overall, the economic analysis of the project at closing indicates that the project is economically viable, considering the actual disbursements and achievements.

41. The budget utilization shows that there was disproportionate use of funds across components compared with the initial planned budget. There were several reasons for this: some activities were ambitious, some were understated in the PAD, and others could not be implemented due to the country’s regulatory framework, for example, open data portal development. Moreover, the plan to implement an e-bus was canceled because it was too difficult to procure a specialized bus in the country and to procure internationally due to small quantity (procurement of one bus) and specialized specifications. As a result, this activity was replaced by other dissemination activities, including infographic competitions and statistical awareness activities on television, radio, and other media. Another reason for project budget reallocation during implementation was because the Governments priorities in statistical support become clearer during project implementation, i.e., systematic institutional capacity building activities (including improving physical and human capacity). These priorities were directly beneficial in catalyzing the overall statistical activities in the NSS.

42. Central Statistics Agency with the support of the SFR project organized a large-scale training on data quality and sampling techniques with low cost and shorter time through the Woreda Net video conferencing system to the NSS members who were at different locations in Ethiopia. This training through Woreda Net was very useful and cost-effective.

43. Increased SCI score. As mentioned above, at project approval in 2014, the Ethiopia Statistical Capacity Index (SCI) was 61.11. The SCI consistently improved after the project became effective in 2014 and relatively higher scores in 2018 (SCI 72.22). The increased SCI score attributes to the CSA’s and NSS’s improved data production and the capacity-building activities supported by the project.

Table 2. Project Actual Expenditure by Components

Sl. No. Project Component Allocated

Budget (US$, millions)

Actual Expenditure

(US$, millions)

Percentage of Expenditure

1 Component 1: Organizational and Infrastructure Capacity Development in the NSS

7.8 11.90 152.56

2 Component 2: Statistical Data Development and Management

3.2 1.63 50.94

3 Component 3: Data Quality Assurance & Information Dissemination

2.4 0.41 17.08

4 Component 4: Monitoring & Evaluation 2.1 1.49 70.95 Total 15.5 15.43 99.50


Page 19 of 45

D. JUSTIFICATION OF OVERALL OUTCOME RATING

44. The overall outcome rating is determined based on a combined assessment of project relevance, efficacy, and efficiency ratings. Based on the above assessment and ratings, with the relevance of the PDO rated High, the efficacy of achievement of PDO rated Substantial, and efficiency rated Substantial, the overall outcome of the project is rated as Satisfactory.

E. OTHER OUTCOMES AND IMPACTS (IF ANY)

Gender

45. At the time of project implementation, the CSA was aware of the importance of including women in project benefits by adding a gender dimension to report under intermediate outcome indicators. Therefore, during the project implementation period out of 4584 staff (1299 women) staff from CSA headquarters and branch offices and NSS members received trainings locally and internationally.

46. The project had supported a further analysis of the 2013 Time Use Survey. Time use survey captured information on individual level time allocation. This allows researchers, program managers and policy makers to look at gender differences in time use, etc. and their effect on other outcomes. Such analysis is also new to the country. By facilitating this effort, the project had made contributions in strengthening gender statistics.

Institutional Strengthening

47. The project sought to undertake organizational and infrastructure capacity development in the CSA. The key activities focused on improving organizational performance, staff productivity, and data accessibility by strengthening planning and physical infrastructure at the CSA. The construction of six CSA branch offices out of twenty-five CSA branch offices and the furbishing of new branch offices with furniture and IT equipment helped to improve the working environment of CSA staff. An IT training room, CSA HQ conference hall, and library are functional at the CSA new building for training of CSA staff. Over 3140 staff (1273 females) from the CSA and MDAs participated in local training (such as statistical awareness, sampling techniques, and statistical software), while over 61 NSS staff (12 females) have benefited from experience sharing training abroad. In addition, the CSA used media and its new website to raise statistical awareness among the public and improve the dissemination of statistical information.

48. Through the support of the SFR, the NSS is also better coordinated. This is demonstrable by the fact that the CSA was able to get additional resources to help build statistical capacity in the country. One of these resources is that the ESPES project funded by World Bank where statistical surveys are used as disbursement linked indictors (DLIs). The CSA continued and expanded collaboration with the World Bank and other development partners, such as, United States Agency for International Development (USAID), Food and Agriculture Organization (FAO), and the African Development Bank (AFDB). The support by the SFR also helped strengthen the capacity of the MDAs.

Mobilizing Private Sector Financing

49. The project did not mobilize private sector financing.


Page 20 of 45

Poverty Reduction and Shared Prosperity

50. The project was not designed to have a direct impact on poverty reduction and shared prosperity. However, strengthening the capacity of the NSS to produce and disseminate quality data and statistics has the potential of improving the evidence base for development interventions intended to reduce poverty and promote shared prosperity. As noted earlier, effective policy making requires quality data and statistics. To fight poverty in all its dimensions and reduce inequality, policy makers need credible information on the nature and extent of poverty and inequality, their root causes, and social consequences. Reliable measures of poverty and inequality are needed to monitor changes at both the individual and societal levels, compare the relative situation of policy-relevant socioeconomic groups, and assess the impact of policy interventions on the target population.

Other Unintended Outcomes and Impacts

51. None.

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME

A. KEY FACTORS DURING PREPARATION

52. The project design was based on extensive consultations with the CSA, the Government, MDAs, and stakeholders. The project design incorporated lessons learned from various statistical development projects in country and other African countries. There was a growing consensus to strengthen the NSS in the production and dissemination of timely and quality statistics relevant for evidence-based policy making and other uses.

53. Lack of clarity during the project design. This is related to the proper identification of outcome indicators with output indicators. As a result, some difficulties arise to measure the project end targets at the start of the project implementation. As a result, the results framework was revised during project implementation in order to capture the project development outcomes.

54. The project design was illustrative in proper identification and description of project components and subcomponents, but the activities identified under each component were not clearly stated in detail to support the components and subcomponents. Some activities were ambitious, some were understated in the PAD, and others do not align with the country’s policies, such as open data access.

55. The PAD discussed the risks that the project would face and identified adequate mitigation measures. Procurement and financial management arrangements of the CSA were thoroughly assessed, and action plans were prepared to mitigate risks.

56. Adequate risk and mitigation measures were identified and implemented throughout the project’s duration. For example, the capacity of the CSA to carry out the project was identified as a risk during project design. This was planned to be mitigated through close monitoring and follow-up by the World Bank project task team; provision of technical visits for CSA managers to good practice locations to gain hands-on experience; and provision of physical infrastructure, technical assistance, and training.


Page 21 of 45

Additionally, to provide CSA project implementation staff with World Bank procurement and financial management training to carry out the project activities.

57. The branch office construction activities were also identified as a risk partly because the construction of the six CSA Branch Offices might have affected the overall timing of the project due to increased administrative transactions triggered through procurements and installation of the ICT infrastructure. The other risk identified with the construction of these offices was the production of noise and waste in the surrounding areas. This was planned to be mitigated by following Government and World Bank safety requirement procedures for construction to reduce the possible impacts and bringing appropriate experts on the team at all levels of the project. During project design, OP/BP 4.01 (Environmental Assessment) policy was triggered and therefore an ESMF was prepared and disclosed before project appraisal and published on both CSA and the World Bank’s website. During implementation, the client prepared and published an Environmental and Social Management Plan for the construction activities.

B. KEY FACTORS DURING IMPLEMENTATION

Factors Subject to the Control of Government and/or Implementing Entities

58. In the initial year, the project faced start-up challenges related to procurement planning and the setup of the PIU. There were also delays in site allocation for the construction of the first four new branch offices.

59. There was overall strong support from the Government including the CSA’s management at the HQ and branch offices, MOF, and Planning and Development Commission. The PIU regularly produced progress reports summarizing recent developments, including information on budget, capacity building, data production and dissemination, and procurement and financial management. The Government communicated regularly with the World Bank to discuss implementation issues as needed and a joint Government/Development Partner (DP) Steering Committee/National Partnership Group was established and provided the national guidance and oversight for the implementation of the SFR Project.

60. However, delays were caused in relation to construction works, including approval of construction-related documents, and provision of utilities, including electric power and water supply and Internet service, etc. During the beginning of project implementation, the PIU did not have experience in management of construction works. Additionally, the engaged contractors had low capacity, which caused an overall delay in the construction activities. However, after the AF, the PIU was more experienced and more efficient in its construction works management capability.

61. Poor Internet connectivity. Poor Internet connectivity adversely affected the project to deliver trainings through the video conferencing system. Even though the achievements registered in the provision of trainings through video conferencing were successful in terms of cost and time, the result could have been even better if the Internet connectivity was more reliable. Poor Internet connectivity also caused delays in updating the Procurement Plans and uploading of the actual status of procurement activities in the World Bank’s Systematic Tracking of Exchanges in Procurement (STEP).


Page 22 of 45

62. There were challenges in procurement, mainly due to challenges in preparing specifications, Bill of Quantities and terms of reference for complex activities. Additionally, local suppliers were not as receptive as required by the project and several bids were cancelled, creating delays in item deliveries. Procurement documents were sometimes not properly documented and there was also absence of proper documentation of the decisions in the procurement process.

63. Shortage of foreign exchange has also been a significant challenge during the procurement process for National Competitive Bidding.

Factors Subject to the Control of the World Bank

64. The World Bank’s regular communication with the client was effective. World Bank project team provided regular on hands training to the PIU in the areas of FM, STEP and Safeguard issues. As a result, smooth withdrawal applications and approvals were done through Client Connection and procurement was done through STEP. The use of these systems provided opportunities for fast and convenient services while ensuring proper documentation of project transactions from the World Bank side.

Factors Outside the Control of Government and/or Implementing Entities

65. The increasing rate of inflation and the devaluation of local currency had affected the performance of the project. Though the devaluation of the local currency resulted in some increase in nominal value of Ethiopian birr, the inflation rate of the goods in the local market could not compensate the gains obtained from the foreign currency.

66. The change in the price of goods had frequently imposed a change on the planned budget and resignation of suppliers in the process of procurement had resulted in change of quantity of items and cancellation of some project activities.

67. In the project period, there were mass movement and a series of civil unrest in many parts of the country and that had affected the implementation of the project by limiting the proper implementation and supervision of the construction activities. As a result, the contractors’ efforts to work according to the contractual schedule was affected.

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME

A. QUALITY OF MONITORING AND EVALUATION (M&E)

M&E Design

68. Some of the indicators in the original Results Framework lacked baseline data and/or realistic targets. As a result, it was difficult to monitor the project implementation progress in the early stages of the project. After the first project restructuring, PDO and IP level indicators had clear target values and M&E became more systematic.

69. Tracking progress toward the desired outputs and outcomes was supported by M&E activities. These included regular project progress reports prepared jointly by the PIU; minutes of meeting of the


Page 23 of 45

National Partnership Group; Aide Memoires of supervision missions, ISRs; and the interim financial reports, audit reports, and the MTR report. Actual values for PDO and IP indicators of project were updated regularly during the preparation of the ISRs.

M&E Implementation

70. The PIU regularly produced progress reports, summarizing recent developments, including information on budget, capacity building, data production and dissemination, and procurement and financial management. Such reports emphasized the key implementation challenges and provided status update on all issues, recommendations, and progress on actions agreed at previous implementation support missions (ISMs) through ISRs and Aide Memoires.

M&E Utilization

71. As noted earlier, project progress indicators were reported in the ISRs and were used to provide evidence of achievement of outcomes.

Justification of Overall Rating of Quality of M&E

72. The overall rating for quality of M&E is Substantial, considering that required M&E staff were in place and carried out periodic reporting.

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE

73. The project triggered environmental safeguard policies: Environment Assessment (OP 4.01). As per the PAD, the disclosed ESMF document defines the management procedures that would allow the proposed SFR Project to “avoid, mitigate, or minimize adverse environmental and social impacts” of supported activities. It was prepared in accordance with definitions provided in the World Bank Operational Manual and in relation to the triggered operational policy on Environmental Assessment (OP 4.01). The other operational policies included in the World Bank safeguards procedures were not triggered by the SFR Project. As the environmental and social impacts of the proposed project activities were not significant, the SFR Project was potentially category ‘B’. Environmental Impact Assessment was found that the anticipated impacts would be short term, site specific, confined, and reversible and could be adequately managed through the proposed Environmental Management Plans.

74. The project has a satisfactory environmental safeguards performance as various activities were undertaken in compliance with the requirements of the ESMF. Among other things, an environmental clause for construction contractors was annexed with the construction contract agreement and signed by the contractors so that the guideline could be legally binding and so that the constructors could take mitigation measures as per the environmental clause. The branch offices regularly monitored the implementation of mitigation measures. However, the contractors were not been formally reporting to the branch offices about the actions taken in compliance with the requirements of the environmental clause. The client has been preparing reports regarding the activities carried out to meet the safeguards requirements of the project. To manage liquid wastes which will be generated at the operation phase of the building, septic tank has been constructed. It was also reported that appropriate soil testing and validation was made before beginning of construction so that the building could be built at a site of


Page 24 of 45

desirable geological quality and stability. Construction wastes have been carted away to avoid the accumulation of wastes at the site despite some irregularities. The project workers reported that the contractor has been providing the appropriate personal protective equipment (PPE). No occupational health and safety incident and no outstanding environmental safeguards issue were reported or recorded in the course of implementation of the Project. Therefore, overall social safeguard implementation and compliance was Satisfactory.

C. BANK PERFORMANCE

Quality at Entry

75. Strategic relevance and approach. The project design reflected the strong interest and commitment of the Government (including the CSA, MDAs, and NSS members) to improve the provision of reliable and timely statistical data. The project was strategically relevant as it was aligned with the GTPII priorities and was closely related to the goals outlined in the World Bank’s 2013–2016 CPS of Ethiopia. Its design aimed to support NSDS, which was expected to provide the relevant indicators to be used in formulating, updating, monitoring, and evaluating the strategies and targets of the country’s social and economic development programs. Therefore, the project design reflected the GoE’s strong interest and commitment to address the challenges of the demand for statistical information in the country.

76. The technical design of the project was based on a detailed institutional analysis of CSA and the related entities comprising the NSS. The World Bank’s procurement and technical experts reviewed specifications for construction works, purchase of computing and communications equipment, business process reengineering plans, and human resource training needs. Least-cost solutions were considered during the design of the project’s activities while marginal financial returns were expected from the project.

Quality of Supervision

77. Supervision of the SFR Project was instrumental to achieve the PDO. All the three restructurings and an AF were necessary to achieve the project outcomes and achieving its objectives. The team provided hands-on support by sharing knowledge on how similar project activities are generally managed in Ethiopia and other countries. For example, infographic competitions were started in Ethiopia, learning from a similar dissemination activity conducted in Rwanda. Introduction and implementation of ICT in survey activities were also conducted through strong World Bank guidance and support.

78. The World Bank ensured that project implementation was carried out according to the project specifications spelt out in the project financing agreements. As evidenced by the series of ISRs produced, the World Bank team conducted regular supervision missions and made efforts to resolve many of the issues following the project implementation. This was also done through regular project updates and periodic project implementation support missions (ISMs). There were nine ISMs during the project life (2014–2019), an MTR, and one ICR mission at the end of the project. The ISMs evaluated progress of implementation as well as challenges and offered suggestions to improve implementation. Aide Memoires issued at the end of ISMs provided valuable information to guide project execution particularly on the status on implementation and the action plans put in place to redirect/realign activities with clear and candid assessments made throughout the lifetime of the project if needed. In addition, the core World


Page 25 of 45

Bank project team were mostly based in the country including the task team leader, the senior financial management specialist, and the senior procurement specialist. This provided constant timely support and follow-up with project implementation and improved communication with the CSA and beneficiary MDAs.

79. Project Closing Workshop was held on September 23 and 24, 2019 at the newly built conference hall at the CSA HQ. The workshop brought together close to 150 participants drawn from NSS members and Donors. The workshop aimed: (i) celebrating achievements of SFR Project; (ii) sharing challenges faced and lessons during SFR project implementation; (iii) identify ways of and responsibilities for ensuring sustainability of the project outcomes; and (iv) statistical data dissemination. In addition, during the workshop a World Bank team discussed how the LSMS-Integrated surveys on Agriculture (ISA) program created a platform for conducting experiments on survey methods to improve data collection in household surveys in developing countries.

80. Overall quality of supervision is therefore rated Satisfactory.

Justification of Overall Rating of Bank Performance

81. Project preparation including identification, design, and appraisal of project were successful and the project achieved the planned development outcomes including the strategic relevance and approach, technical, financial, and economic aspects.

82. The implementation support by the World Bank team during project implementation is rated Satisfactory. Many of the final project activities would not have been implemented and objectives would not have been achieved without the level of implementation support provided by the World Bank project team.

D. RISK TO DEVELOPMENT OUTCOME

83. During project preparation, the project design considered some possible risks and assumptions. Among these were (a) the CSA’s limited experience to coordinate with MDAs; (b) The capacity of the CSA management to carry out activities identified in the SFR project. These issues were addressed through the provision of technical experts, funding of annual statistical conferences and technical visits of CSA managers for experience sharing to Mexico statistics to gain hands-on experience.

84. As part of sustainability, ESPES project will continue financing CSA on indented project activities. However, there remains a risk that after the SFR project closure, there would be a possible financing gap in relation to capital expenditure such as, repair and maintenance expenses of infrastructure investments, including the newly built six branch offices, HQ conference center, training center renovated library, and ICT improvements and the regular user satisfaction surveys that have been conducted with the support of the project. In addition, the risk of erosion of Adama branch office building fence. The Adama Branch office building is very close to a river course which is quickly approaching the fence and the compound. Considering the sandy nature of the soil, it requires serious attention and conservation before it creates damage. The projects task team has already discussed with the World Bank’s Urban Productive Safety Net Project (P151712) project implementation unit to help construct gabions around the river course to avoid any damages that might be caused during the rainy season. The same information was communicated with the CSA.


Page 26 of 45

85. Even though the ESPES Project might lessen the risk, future budgetary support to statistics by the Government and Donors should be made to help the CSA and the NSS improve statistical data production, quality, and dissemination through various capacity-building activities including human resource development, support of survey activities, and purchase of IT equipment.

86. The CSA and project team agreed that no major issues or risks are anticipated besides the above-mentioned issues.

V. LESSONS AND RECOMMENDATIONS

87. Having client commitment, ownership, and accountability around a strengthened NSS is key for the achievement of results. The CSA has demonstrated a high level of commitment to statistical development. There was (a) a continued understanding of project objectives and support for its success, related decision making, and management of the project; (b) clear understanding of how the project was helping in development of the institution; and (c) deeper knowledge of the broad sector reform process.

88. Risk to sustainability of project results. The project helped improve the infrastructure of the CSA branch office and HQ offices. It is recommended that the Government should allocate resources for the operationality, and maintenance of these facilities. For example, the Video Conference centers are functional but have not become fully operational yet. The agency still has to get used to these modern ICT technologies and start using them on a day to day basis. The buildings should have yearly maintenance budgets from the Government, and positions including general service to maintain the buildings should be hired at each new branch office site. The training centers at the HQ should also take advantage of the space and facility to regularly implement training and video conferencing activities. The CSA should also aim at making an organizational/institutional change so that it can keep up with the increasing demand for better and timely data for evidence-based policy decision making.

89. Ensure continuous professional training to improve performance as well as provide special attention on implementing scheduled activities.

90. Operationalization of modern ICT services. The video conference centers established at the CSA HQ and four new branch offices are expected to be properly managed and effectively utilized to provide regular trainings and conduct seminars and workshops at a larger scale. The CSA should encourage its staff to use the video conferencing facilities for its intended purposes.

91. Improving selection criteria of trainees so as to include more female participants. It is recommended that CSA to revisit the selection criteria of trainees of NSS staff and make sure women are fairly represented.

92. Engagement of NSS members across all project activities. The engagement of NSS members (both at the federal and regional bureau level) was limited, especially in IT-related activities. Hence, the involvement of all the NSS members should be ensured in future capacity-strengthening activities, especially in areas of IT infrastructure development. Additionally, future statistical capacity building projects are encouraged to support the CSA in its responsibility to lead and coordinate the NSS.


Page 27 of 45

93. Results Framework Design: The project’s Results Framework was changed in order to better align the project activities with the priorities of the client and adjust to unforeseen circumstances. However, future similar projects should give special attention during the preparation of the project to better understand the priorities and capacity of the implementing agency and the rules and regulations of the country. The Results Framework should also be based on measurable targets that are realistically achievable in the medium term, rather than longer term outputs.

. .


Page 28 of 45

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS

A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Enhance the capacity of the CSA to produce and disseminate reliable, accessible, & timely statistics

Indicator Name Unit of Measure Baseline Original Target Formally Revised Target

Actual Achieved at Completion

Indicator 1: Key NSS members who have applied the EDQAF manual

Number 0.00 0.00 5.00 5.00

30-May-2014 30-May-2014 15-Jun-2017 30-Sep-2019

Comments (achievements against targets): These NSS members include Ministry of agriculture, education, health, road and water resources.



Indicator 2: Percentage of Percentage 0.00 97.00 97.00 80.00


Page 29 of 45

users who are satisfied with official statistics methodologies as determined by the user satisfaction survey.

30-May-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019

Comments (achievements against targets):



Indicator 3: Execution rate of annual statistical release calendar

Percentage 0.00 75.00 75.00 71.00

30-May-2014 30-May-2014 15-Jun-2017 30-Sep-2019




Indicator 4: Percentage of users who are satisfied with the accessibility of official statistics as determined by the user satisfaction survey

Percentage 0.00 94.00 94.00 77.00

30-May-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019



Page 30 of 45

A.2 Intermediate Results Indicators

Component: Component 1: Organizational and Infrastructure Capacity Development in the NSS



Number of new branch offices buildings that are operational

Number 0.00 4.00 6.00 6.00

30-May-2014 30-May-2014 15-Jun-2017 30-Sep-2019




Conference Hall and a Training Center established and operational at CSA Headquarters

Yes/No N Y Y Y

30-May-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019





Page 31 of 45

Number of video conference centers established and operational

Number 0.00 6.00 6.00 6.00

30-Apr-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019




Number of staff from CSA and MDAs trained annually on areas of organizational management, IT, and Statistics

Number 0.00 1525.00 1525.00 1456.00

30-May-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019




Number of female staff from CSA and MDAs trained annually on areas of organizational management, IT, and Statistics

Number 0.00 534.00 534.00 542.00

30-May-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019



Page 32 of 45

Component: Component 2: Statistical Data Development and Management



Statistical Business Register developed and operational

Yes/No N Y Y Y

30-May-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019




Number of surveys supported by the project

Number 0.00 6.00 6.00 6.00

30-May-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019




Number of people trained on areas of statistical data production

Number 0.00 387.00 387.00 198.00

30-May-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019


Page 33 of 45

Comments (achievements against targets): Component: Component 3: Statistical Methodology, Standards and Data Quality Assurance and Information Dissemination



Number of CSA and MDAs staff trained in specialized quality assurance topics including EDQAF survey methods, poverty mapping techniques, gender analysis of household surveys

Number 0.00 2944.00 2944.00 2930.00

30-May-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019




Number of female CSA and MDAs staff trained in specialized quality assurance topics including EDQAF survey methods, poverty mapping techniques, gender analysis of household surveys

Number 0.00 765.00 765.00 719.00

30-May-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019


Page 34 of 45




User-friendly CSA website established

Yes/No N Y Y Y

30-May-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019




Number of user satisfaction reports

Number 0.00 3.00 3.00 3.00

30-May-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019

Comments (achievements against targets): Component: Component 4: Monitoring and Evaluation



Finalize NSDS II Yes/No N Y Y Y


Page 35 of 45

30-May-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019




CSA project coordination team and PIU established and operational

Yes/No N Y Y Y

30-May-2014 15-Jun-2017 15-Jun-2017 30-Sep-2019




Number of audit reports produced

Number 0.00 4.00 5.00 5.00

30-May-2014 15-Jun-2017 19-Apr-2019 30-Sep-2019




Number of project Number 0.00 2.00 2.00 2.00


Page 36 of 45

evaluation reports produced 30-May-2014 15-Jun-2019 15-Jun-2017 30-Sep-2019



Page 37 of 45

ANNEX 2: KEY OUTPUTS BY COMPONENT

Objective/Outcome 1: Reliability and Quality:

Outcome Indicators 1. Number of key NSS members who have applied the EDQAF manual

2. Percentage of users who are satisfied with official statistics methodologies as determined by the user satisfaction survey


1. Number of new branch office buildings that are operational 2. New conference hall and training center established and operational at CSA HQ 3. Number of surveys directly supported by the project 4. Statistical Business Register system developed and operational 5. Number of female CSA and MDA staff trained on specialized quality assurance topics including EDQAF, survey

methods, poverty mapping techniques, gender analysis of household surveys 6. Number of user satisfaction survey reports 7. Number of CSA and MDA staffs trained annually on areas of organizational management, IT and Statistics 8. Number of Female CSA and MDA staffs trained annually on areas of organizational management, IT and Statistics

Key Outputs by Component (linked to the achievement of the Objective/Outcome 1)

Component 1 1. Fully operational six branch offices in Mekelle, Hawassa, Bahir Dar, Ambo, Adama, and Dessie with new services,

comprising training rooms, mini-data centers, and IT kiosks for the storage of statistical information; mini-libraries; and space to accommodate additional staff

2. New conference hall built, training center, and library renovated at the CSA HQ 3. IT facilities including Internet connectivity upgraded and functional in CSA HQ and in newly built four branch

offices 4. Equipment including desktops and laptops delivered to CSA HQ and branch offices 5. 1,456 staff from the CSA and MDA trained annually on areas of organizational management, IT, and Statistics 6. 542 female staff from CSA and MDA trained annually on areas of organizational management, IT, and Statistics Component 2 1. Support to Surveys: Support to Agriculture Surveys through the procurement of 2,300 digital weighting scale

and 2,300 measuring taps; Support to Population and Housing Census Pilot Survey by introducing CAPI tablets;


Page 38 of 45

Support to Statistical Business Survey; and Support to Household Consumption and Expenditure Survey through trainings

2. Cartography and GIS departments supported through the procurement of GPS-enabled data loggers, Arc GIS software, and 400 solar panels

3. Statistical business register system developed and operational 4. 198 CSA HQ and branch office staffs trained in areas of statistical data production Component 3 1. 2,930 CSA and MDA staffs trained on specialized quality assurance and related topics 2. 719 female CSA and MDA staffs trained on specialized quality assurance and related topics 3. Three user satisfaction surveys conducted

Objective/Outcome 2: Timeliness

Outcome Indicators 1. Execution rate of annual statistical release calendar


1. Number of surveys directly supported by the project 2. Statistical Business Register system developed and operational 3. NSDS II finalized and approved 4. Number of people trained in areas of statistical data production


Component 1 1. IT facilities including internet connectivity upgraded and functional in CSA HQ and newly built four branch offices Component 2 1. Support provided to surveys: Agriculture surveys through the procurement of 2,300 digital weighting scale and

2,300 measuring taps; Population and Housing Census pilot survey by introducing CAPI tablets encouraging CSA to use CAPI; Statistical Business Survey; Welfare Monitoring Survey through provision of trainings and study tours to the CSA HQ and branch office staffs and to Commission for Planning and Development staff; Household Consumption Expenditure Survey through trainings

2. Cartography and GIS departments supported through the procurement of GPS Enabled data loggers, Arc GIS software and 400 solar panels

3. Statistical business register system developed and operational 4. 198 CSA HQ and branch office staffs trained in areas of statistical data production


Page 39 of 45

Objective/Outcome 3: Data access

Outcome Indicators 1. Percentage of users who are satisfied with the accessibility of official statistics as determined by the user satisfaction survey


1. Number of new branch offices buildings that are operational 2. Conference hall and training center established and operational at CSA HQ 3. User-friendly CSA websites established 4. Number of video conference centers established and operational


Component 1 1. Fully operational six branch offices in Mekelle, Hawassa, Bahir Dar, Ambo, Adama, and Dessie with new services,

comprising training rooms, mini-data centers, IT kiosks for the storage of statistical information; mini-libraries; and space to accommodate additional staff

2. New conference hall built, training center and library renovated at the CSA HQ 3. IT facilities including internet connectivity upgraded and functional in the CSA HQ and newly built four branch

offices; Component 3 1. CSA log rebranded 2. CSA website developed 3. Data dissemination supported through media, publishing of survey reports, meetings and workshops. Component 4 1. NSDS II finalized and approved


Page 40 of 45

ANNEX 3. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION

A. TASK TEAM MEMBERS Name Role

Preparation

Supervision/ICR

Tom Bundervoet Task Team Leader(s)

Waleed Haider Malik Task Team Leader(s)

Senait Kassa Yifru Team member

Asferachew Abate Abebe Social Specialist

Pascal Tegwa, Rahmoune Essalhi, Shimelis Woldehawariat Badisso

Procurement Specialist(s)

Mekdim Hailu Yemane Financial Management Specialist

Biftu Wordofa Team Member

Jean Okolla Owino Team Member

Martin Pedro Buchara Team Member

Stella Chepkorir Team Member

Melat Assefa Asfaw Team Member

Abiy Tessema Ashagre Team Member

Candace Asrat Gebre Team Member

Moges Woyessa Bowuso Social Specialist

Samuel Lule Demsash Social Specialist

Clara Maghani Team Member

Tamru Demsis Temam Environmental Specialist


Page 41 of 45

B. STAFF TIME AND COST

Stage of Project Cycle Staff Time and Cost

No. of staff weeks US$ (including travel and consultant costs)

Preparation FY19 0 1,125.00

Total 0.00 1,125.00

Supervision/ICR

FY14 0 9,929.43

FY15 1.988 59,154.03

FY16 6.750 68,985.53

FY17 16.088 144,377.60

FY18 16.544 138,640.16

FY19 9.650 133,286.75

FY20 13.145 135,507.49

Total 64.17 689,880.99


Page 42 of 45

ANNEX 4. PROJECT COST BY COMPONENT

Components Amount at Approval

(US$, million)

Amount after AF and Restructuring

(US$, millions)

Actual at Project Closing (US$,

millions)

Actual Percentage at

Project Closing

Component 1: Organizational and Infrastructure Capacity Development in the NSS

4.90 7.80

11.9 152.00

Component 2: Statistical Data Development and Management 1.50 3.20

1.63 150.94

Component 3: Statistical Methodology, Standards and Data Quality Assurance and Information Dissemination

1.90 2.40 0.41 17.08

Component 4: Monitoring and Evaluation 1.70 2.10 1.49 70.95

Total 10.00 15.50 15.43 99.50


Page 43 of 45

ANNEX 5. LIST OF TRAINING BY TOPICS, LOCATION, AND NUMBER OF TRAINEES BY YEAR

Sl.

No. Training Areas Location Year Number of Trainees

Male Female Total A. Local Trainings

1 Computer training Ethiopia 2016 18 40 58 2 Computer training Ethiopia 2017 240 232 472 3 Computer training Ethiopia 2019 40 37 77 4 Training on financial

management Ethiopia 2016 130 118 248

5 Safety and maintenance for drivers

Ethiopia 2017–19 97 2 99

6 Leadership and change management

Ethiopia 2017 23 3 26

7 IPEX Training for CSA financial experts

Ethiopia 2018 30 37 67

8 IPEX and auditing training for CSA auditors

Ethiopia 2018 26 12 38

9 Stata training for CSA staff Ethiopia 2017 21 1 22 10 Stata training for CSA staff Ethiopia 2018 93 11 104 11 CSPro training for CSA staff Ethiopia 2018 103 11 114 12 CSPro training for CSA staff Ethiopia 2019 19 4 23 13 Basic computer and

Photoshop training for CSA staff

Ethiopia 2019 0 19 19

14 Support for MSc. study Ethiopia 2016–19 34 7 41 Total Component 1 Ethiopia 874 534 1,408

15 Agricultural survey software training

Ethiopia 2016 74 17 91

16 Statistical package training for national account staff

Ethiopia 2018 7 6 13

Total Component 2 Ethiopia 81 23 104 17 Video conference training on

statistical awareness Ethiopia 2016 157 52 209

18 Video conference training on statistical awareness

Ethiopia 2016 728 300 1,028

19 Video conference training on statistical awareness

Ethiopia 2017 1,195 331 1,526

20 Training on sampling techniques and survey method

Adama/Ethiopia 2016 65 15 80

21 Training on gender time use survey in-depth analysis

Addis Ababa/Ethiopia 2017 14 7 21

22 Training on web portal and e-learning

Addis Ababa/Ethiopia 2019 14 6 20

23 Training on website design Addis Ababa/Ethiopia 2016 4 2 6 24 Training on statistics

communication for media experts

Bahir Dar/Ethiopia 2017 8 3 11


Page 44 of 45

Sl. No. Training Areas Location Year Number of Trainees

Male Female Total Total Component 3 2,185 716 2,901 Total Local Training 3,140 1,273 4,413

B. International Training 25 Advanced project

management Dubai 2016 4 — 4

26 Training on World Bank Financial Management Manual

Dubai 2017 4 1 5

27 Training on World Bank Procurement Manual

Dubai 2017 4 1 5

28 Training on video conferencing system development and operation

Dubai 2018 1 1 2

Total Component 1 13 3 16 29 GIS application Nairobi 2015 12 1 13 30 Database and web GIS

application Thailand 2017 4 1 5

31 Training on national accounts system development

Thailand 2016 5 3 8


Thailand 2017 2 4 6


Thailand 2018 3 1 4

34 Training on field operation India 2017 14 — 14 35 Training on field operation India 2017 12 — 12 36 Training on business statistics Netherlands 2017 4 1 5

Total Component 2 56 11 67 37 Poverty analyses Thailand 2016 10 — 10 38 Data quality and

standardization Thailand 2016 5 — 5

Total Component 3 15 — 15 Total Abroad Training 84 14 98 C. International Study

Tour and Experience Sharing Visit

39 Experience sharing on statistics bureau operation and management

Canada 2017 1 1 2

40 Experience sharing on fast track construction

Tanzania 2016 3 0 3

41 Experience sharing on Mexico statistics

Mexico 2016 2 0 2

42 Experience sharing on Thailand statistics

Bangkok 2018 4 1 5

43 Experience sharing on IT database system

Indonesia 2017 4 1 5

44 Experience sharing on IT application

United States 2016 1 0 1


Page 45 of 45

Sl. No. Training Areas Location Year Number of Trainees

Male Female Total 45 Experience sharing on NSDS

Development Indonesia 2015 5 1 6

46 Experience sharing on NSDS Development

Malawi 2015 5 0 5

47 Experience sharing on IT application

South Africa 2015 2 1 3

Total Component 1 27 5 32 48 Experience sharing on

agriculture, business population, and price statistics

Republic of Korea 2017 8 2 10

49 Experience sharing on fishery statistics

Indonesia 2017 5 0 5

50 Experience sharing on GIS-GPS application

Cabo Verde 2015 4 1 5

51 Experience sharing on vital statistics

Botswana 2016 6 1 7

Total Component 2 23 4 27 52 Experience sharing on user

satisfaction survey Rwanda 2015 4 1 5

53 Experience sharing on post enumeration survey and IT database system

South Africa 2017 7 2 9

Total Component 3 11 3 14 Total Study Tour and

Experience Visit 61 12 73

Grand Total 1 (A + B + C) 3,285 1,299 4,584

World Bank Document - Clearinghouse for Financing ...

Documents

Transcript of World Bank Document - Clearinghouse for Financing ...